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ABSTRACT 

This Bonograph coBpris<=>5; five papers presented at a 
synposiuM whose primary purpose was to ^ scuss the role of direct 
observatiohal techniques in the study of certain learning outcomes 
and of certaih learning and instruction process variables in school 
settings. Each study represents an attempt to apply direct 
observational techniques to the investigation of the 
instructional-learning processes and tq use the observational data 
for evaluating and documenting a vide rang^e of learning outcomes. "Lhe 
first two papers, each examining a dif€erent technique* focus on how 
observational techniques are used as an evaluative instrument^ to 
study classroom behaviors across diffujse geographical locations 
and/or across different educational programs. In the-three succeeding 
papers, the authors report studies desigi^ed to focus on certain 
"instructional-learning behaviors for a particular purpose other than 
for the evaluation of classroom behavior in a summative sense. The 
last section contains two discussions that offer critical comments on 
the studies reported and suggest^ impl inflations for farther research. A 
"Description of the LearniQjg Research knd Development Center's 
Individualized Instructional Programs^iy definitions; and a 
description of the observational instrument and techniques, data 
recording forms, and the criteria for t.l^eir use are appended. (PD) 
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^ • 3EST copy AVAIUIBil , 

FOUI^WORD ^ 

Vl'«'^* ^ini{. like other professtona 1 endeavors, restd upon a fnunda- 
tian of arc inuflated knowledj^c and artistry o( the practitioner. Any ahaly- 
sin of ihe It achitiL' process and the activities of teachers must keep both 
knowknl^i .Mid artistry in mind. Attempts to study classroom practices 
nntfif prubc into both ihest.- levels:^ M^^nerai principles that underlie tlie 
Wiiy« which Warning can occur, and nuances of technique imposed by 
ii-acUer .r studcMit behavior that efthor enhance ur deteriorate the Instric 
tiunal rM<»dct bein>4 implemented. In addition, studies of those unexpected 
emerjitftiu nfot;* sses that result when particular practices are carried out 
In a Mh'H>f yi Uini^ are also in ortler. 

• . • 

It I.S .ipparunl that process variables art- thr key to understanding 

\vhal lioi-s ),i in school^s*. In the recent past, educational research studies 
ft*i*qut nily rt*'>orted that usinj^.method A in the classroon'i was more effec- 
tive ihai^ usuia molh( c! !^ or that a comparison of method A and a control 
group Khow'^fl no difference 5n iheir effects. Studies of this kind carefully 
^descrlbt^d (I) the characteristic's of the student population; (2) the kin^l'of 
jeaching variable manipulated (-t\ p , instruction with or without TV, indi- 
vidualijced versus conventional instruction, or a permissive teach<?r as 
com|>ared to a more directive teacher); and (3) student outcomes (e.g., 
achievement or attitude). Conclusions were then drawn about the influ- 
ence of thb intervening^ teaching variable. Upon reading these accounts, 
the nam;inu question that always came to mind, was: What went 'on in the 
claPKrooni ' (What werc^he processes that differentiated TV tcacbinii 
from tea^chin^ without TV. or that distihguistied permissiveness from 
directive instruction ' ) Most studies described the practices that were ' 
devij5ec1 for classroom implementation. But. the details of how these 
practices were carried out were rai'ely s'o carefully analyzed as student 



differences or edi.caliondl outcomob for wlucli loal s wore nlUioi' avail*-; || 
ble or specaliy cU v clop. .). DiOCorenccs in the degror of impltjmcnlaUoftg;^ 
uf Ihei leaching variable that required Iho dosisn of clclailetl x,beervallontt!i| 
procedures wore raroly «pccifiod and asBosaed in any precJnc- fashion, | 
As » consequence <,f Uu'se «tudic3. sonu- .practical dec i sions w«r e made 
for or a«ainsi the teaclunu practice involved, but little vvas/learned aboufc. 
tho .md«M-lyin« processes that intervened between student clmra^:ter5 sties V 
and student ouiconiefs, ' - 

The m(?rit u( this symposium is its en-iph.isis onYho description of 
process variables. In their papers, all authoi« realUe that. Iniprove- 
ment« in leachinj- require cumulative knowledge based.upon the careful 
observation of instruct mnal - learning processes in school settings. ToH 
a useft.l conlTibulion to cumulative understanding, this knowledge nniSt 
have certain cluvracterisllcs. First, it must be articulat«fd ih terms oi 
generalized conceptual categor'tcs that provide a framework £ot focusing 
obscviAaiion and analysis. In this way, studies of tca6hing caji develop 
richer interconnected theoretical structures than have come outcast 
studies carried cut with few conceptions that arc linked to oiK- another, 

A second characteristic of descriptions of school processes Is 
that they must be treated in context, i.e. , related to student fharac* 
terislics, teacher styles, school settings, and different bducational 
uoa'ls, While leaching processes are widely applicable to the extent 
that Ihey reflect general principles, their application demands that the 
prmciples be adjusted to the conditions of specific situations. The appli- 
cation of a general principle ot scientific law must always Include^paf aw 
eters relevantn©3 the local setting. In their studks of the processes of 
leaching and learning, the authors of the papers in this symposium Sre 
sensitive both (o the requirements of generaliv.ability for knowledge 
accumulation and of specificity for describing actual practice. 
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There are several ways to read this symposiuni. My own reconi* 
mendatlon f§ that the reader /^tart at the^end, with ihc discussants* com- 
metus and observations, and then sample the papers according to his 
{mcHjiallons. lij this way, the reader can get a feeling of the study of 
pirocess from the point of view of his interests, whether they focps on 
the desi^ji of an observational system, observ-?ition as a tool for evaluating ^ 
the implemcniat ion of an instructional models or observation as a way of 
studying emergent social structure in the classroom. ^ • 

- — m 

The papers in this symposium carry us another step toward a 
future in which leachinj; and evaluation research in eJucation will no 
lonfjur be isolated collections of classroom studies but will instead be 
InvcstiKations of class/oom processes based upon systematic conceptually 
based observational schemes. Asa result, groups of studies will relate 
to omi another, and a coherent and improved understanding of the processos 
at wyrkjt classrooms will emerge. 



Robert Glaser* 
Pittsburgh, Pennsylvania 
November^ 19V3 
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INTRODUCTION 



It IS widely .acknowledged that because of the lack of adequate teoh^ 
$ilql}0S« a great number of>4mportant va r La b^s related to schooling-- 
ihstl'Uctlon&I and learning processes, as wel9 as^Jearning outcomes- -are 
Ofteif$ Ignored and seldom systematically studied, Di spite^of the fact that 
in r<Jcent years, educators and educational researchers have been stress - 
irig the need to develop techniques that provide information for those vari« 
abies for which the traditional paper and pencil techniques are neitheV 
sufficient nor appropriate; this traditional practice still prevails. Tho 
^papcrs^"inGluded In this monograph are examples of research studies . 
designed specifically to m<*et the urgent need for the development of • 
altefhative techniques in the investigation of variables r<|^lated to what 
goes on In schools* 

,The papers are proceedings of a symposium, presented at the 
Annual Meeting of the American Educational Research Association in 
New Orleans, Louisiana, February 1973, The primary purpose of the 
symposium was to discuss the role of direct observational techniques in 
the study of certain learning outcomes, and of certain learning* and instruc 
tional process variables in school settin^s^ Each study represents a 
ynique attempt to apply direct observational techniques to the investigva- 
tion of the instructionaUlearning processes, and to use the observational 
data for evaluating and* docurnenting a wide range of learning outcomes. 

The first two papers, each examining a different techn^ique, focus 
on how observational techniques are used as an evaluative instrunjent to 
study classroom behaviors across diffuse geographical locations and/or 
across different educational programs. Both techniques, originally 
designed to assess the degree of Implementation of a particular educa- 
tional program, were used in Project Follow Through, a nationwide 

. M 

' I 



program IniUateci by the United States Office of Education In 1967. ''Ph«j i;^ 
Drogram was Initiated as an effort to improve the education for low inad|i|^ 
children. Through the development of innovative educational ^rOgfs^rtls . 
for the first^four grades in the American public schools (kindergiirteh 
through J4rade three). Project Follow Through attempts toeing about 
educational reforms in the public schools. The* Bank Sfcreel C^l^S^ t^^:^ 
low Through Program, described in the Ross and Zimiles papyri ami 0J 
Learning Research and Development Center (LRJ5C) Follow Through Frr 6*^ 
gram, described in the Leinhardt paper, wera among the fir^t pl*Ograiti§ 
developed Vnder the lede rally -sponsored project, 

The Differentiated Child Behavior pbservation Sy.stem: (DCB), - x 
dcscj*lbed in the paper by Ross and Zimiles, was developed to obtain 
des^'riptive information about clasoroom interactions in order to^ompafl^; 
and differentiate program impacts on students' classroom behavior {c^g^ 
nitlve. affective, physical). Leinhardt, in "Observation Ss a ^ooj for 
^Evaluation and Implementation,** deals 'specifically with the use of' direct 
observation as a means to monitor the implementation of a particular pt^r 
gram. Leinhardt uses the observational data to measure and compare 
differences <\nd similarities among classrooms using the same instrud*- 
lional program. 

In. the three succeeding papers, the authors report studies designed 

W focus on certain instructional-learning behaviors for a particular pur- 

ft * 

pose, other than for the evaluation of classroom behavior in a summative 
sense. Wang reports on a study designed to illustrate the use^ of direct, 
observa-tional j^hniquesjtj^ormati'-e evaluation of an instructional- 
leai:frinj4 management system developed to implement an individualized 
'instructional program, Wa'ng specifically emphasises the application of 
observational data to the development and refinement of the instructional 



J>togr«3im» §bimron repoxis.on a, study designed to investigate th^ offocfs 
oimn educfational program on Indlvldyal students for whom certain stMdi^tii 
leartiing characteristics arc known. Shiairon focust^s on the use 6f obsof^ 
yational data to document l^avior patterns of ^hc Indivislual stud<^nt in 
el^ssroofn Icftrning situations. The papers by Leinhrf i^^lt, Wang, and 
Shlmron are related in the sense that they are concerned with the evalu- 
ation of certain aspects of the same educational prugram, the I^KD.C 
fcstructionai Program. For a brief description of the LRI^C program, 
see Appendix A. 

* Dttiark and^Edelman discuss the application dt ethological concepts 
and methodfi to study the social behavior of young children. The authors 
suggcst^a method, which is well -established In the naturalistic study of 
the social behavior of primates, as one way to obtain a broader and deeper 
study of chUdren^s conct^pts of tlieir social world, and their interrelation- 
ship with peers. 

The la8i«section include* remarks by Medley and Resnlck who were 
dlscusi$nnts for the ^symposium. The remarks offer critical comments on ^ 
the studies reported* and suggest implications for further research in 
l^Tis area. 

it is hop<?d that this monograph, reporting examples of fruitful Use 
of direct observational data /or studying the Instructional-learning processes 
and outcomes, will demonstrate the possibility, of and the need for broad^jft- 
ing the base of school evah'.alion, as wefl as the development of a variety 
of alternative techniques to investigate and evaluate^certain aspects of 
schooling. 



Margaret C. Wang ^ 
Pittsburgh, Pennsylvania 
November, 1973 | 
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^ THE DIFFERENTIATED CHIU5 BEHAVIOR 
^ OBSERVATIONAL SYSTEM 



Sylvia Ross and Herbert Zimiles 
^ank Street College of Education 
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in recont years, researchers have increasingly sought to clfatad- 
trerlxe classroom Interactions in a systematic and relatively objective 
manner. These efforts 'can he justified in terms of^he need for^a des^rip^ 
tlon of the life experience of the child in the classroom. Although it is 
recognl'^ed that the classroom constitutes a significant influentc on the 
chi^cMs development and is a majpr source of each individual's introduce 
tlon and exposure to society, we lack even basic information abojut the 
nature and content of classroom interactions. However, a numojBr of. 
critical issues have |/en<jrated these efforts: (1) tllie rcarch for g-omraLliz" 
able Information about teacher effectiveness and claj^sroom climate that 
could be used both for teacher training and evaluation purposes^ aftd 
(2) the growing discontent with using applied standardized tests for evalu* 
atlrtg educaih>nal programs. The increasing recqgnltion of tho ^imitations 
of lh<?se measures, particularly In reference to assessment of this effects 



The rpsearcn reported in the present form is "suppo rted by a 
Krant from the Tord Foundation to the authors* The' authors would like 
to acknowledge Elizabeth Gilkeson who made substantive contributioris 
diU inK the development of the DCB; Barbara Blber,-, Margery Franklini 
ancj c;arda Bowman for the valuable suggestions and critical comments 
regArding particular.a«pects of the observation system; and Dinahs Heller 
a^id iMlcluiel Moss fcJjf their assistance in supervising the data gathering 
'by a team of graduate students. 

\ 
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of foci e rally -f«4di^d programs on Inner-city children's cognitive ablHti^sir 
has Sttmuitlted a search for broader and more encompassing evaluation 
nml pKOgtam ahalys-is mr^sures. . ^ 

P^.P.r^' f^'^ of the Sys tcirr 

The Differentiated Child Behavior Observational System ^DC13) was ' 
'Origlrially developed under the auspices of a Follow Through sponsor--. 
Bank Street College of Education- -a^s one of a number of self-evamailon 
mMsures designed to assess the extent to which ^its own program hud " 
been successfully implemented. The Bank Street sponsorship of Follow 
Through classes m 14 communltiesjn the United States involves the impie 
mentation In Inner-ciiy public school classroom^ of a "developmental- 
lntf*ractlon, open-classroom approach (see Shapttjo ^< Biber, 1972) 
developed and applied over many decades in the College's School for 
Children. " ^" 

The. complexity of life in the info^iPi;;ml or op*jn classroom makes 

re«^erding of all relevant information difficult. The term "relevant'Siscd 

in this context defines the extent to which choices are* made on the 6a'sis 

of valuer, or cf judgments as to, which behaviors will provide criterion - 

.JlT<?-lsaras of children's classroom behavior. The cont^^nt of the DCB 

^ ft 

instrurtients reflects a set of assuniptions and values underlying the Bank 
Street approach (Gilkeson. 1970), with "competence . . . conceived 
ftinctlonally in terms of hov/ the individual interacts with the challenges, 
the people and the work of his environment" (Biber, ^apiro, Wickens, 
1971). It ^'Iso reflects an attempt toVlelineate a comprehensive and 
detailed roster of typical classroom interactions. The basic assumption 
underlying the design of the DCB is that children's behavior (cognitive, 
affective, social, and physical) r^eflects the attitudes, values, and cur- 
riculum fQ<:i of the classroom instructional team, 
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I'htf Opt n Clarts rooni vcr^u B ihe T r adiU Qnal Cngjij^rot^ 

How rtre, th« differences between an o'h n uml trac^itional clagsroOmH 
refunded 'ift children's interactional behnviur- * What'iS fhe.^ofXoct of 
informal spaliai .ir rangc*mcnts and Kreaior (oachrr and pupilj* mobility Oil J 
tha qimnlily and quality of classroom interactions • Does ai%' •'Opett/' . ' ^ 
l.ndf»pGndcncc-fo5tering, child-ceniercd pnvironinunt, lhat ^pp6S^<ily j 
enco'iranes f*elf-e>&pression. produce a grt-ati-r inc id ence^of destructive/ 
nciU^R-out bcluyior than a stHting which has-a high d^j^ree of control as 
one of its major practices ' Does tlie attempt lo integrate rnd baUnce 
counilive, affccl ijf'e. aesthetic, and social learning experiences vrsuU 
in lert» cpKni.l|lA't*' involvement than that found in traditional settings whei^e 
jjeAll6jyvi<^/leari:UnK is the primary objective * 



The U elation of SES and Ethnic ity to Pro>^ram Patterns 

— ■ '■ •*'<• — •' — - — — , ^ . 

To v/hat extent does the SES and ethnicity of a school population^ 

affect quantity and quality of claFproom interactions ' How does the 

hehavlorof the inner-^city child in the open classroom differ from^fiis 

bt^havior m a traditional setlinj/ ' How does the behavior of thej'^w 

Income child differ from that of the nnddle-cla^.s child in^eith/r setting? 

i 

The Ob s e r v a ti o na j_ Sy s t e m 

The r)CB is iisod "livi-" in llu» cla/ssroom. Data arc j-aHieretl by 
trained observers who oncotic childri-n's interaetlpnal Whaviors op timed 
and chJintfo-of-bc'havior bases. One uf the distinctive features of this 
system Is its emphasis on the snbslantive aspects of children's Inter- 
actions' it provides data re^ardinu the content 'as well as thft sourcfi and 
direction of each entry. In.addition. unli^ many previously designed 
observational approaches, it incorporates a number of procedures that 

6 \ 



are applicable both to informal, open c la$t?es as \vqII/£&' to moro» tradl- 
tiona4 settings. The observatioial system IncUides two Instrumgnls: the 
pC'I^.Form and the Classroo0 Scan. (See Appendix B. ) 

7Jne DC B jFor ni ' ha s been designed to provide quantitative and quali-' 
toU^;«?/daia regarding cKildronV^ ve^rbal and nonverbal classroom behav- 
lorsy X^^^ focui! (whether In observations of smaJl,. large, or total groups) ^ 

is c/n the' number of occurrences of specified behaviors as well as on the 
■ / # i • * • 

nadfre* of the interaction In each instance, i.e., child-to-child, child-to- 
/ ■ ■ • ' 

afdul^,. to or by self; adult-elicslted or child-initiated; individual or choral • 

tespynsc. The referent child's sex is also indicated in each instance* i» ' 

/ ■ " •■ / ■ 

/ The six major behavioral categories of the DCB are: Giv >s Infor- 

/ '■ 
mation, Asksv Questions. Expresses, Acts. Destructively, Organizes and 

<i ^ 
Manages, and Represents and Symbolizes. Each of these six, categories 

iifcludes from six to nine subcategories that are designed to identify spe- 

* 2 ' 

cific behaviors within eadh general category. 
« , c 

The first two categories, ^'Glves Information" and VAsks Questions,'* 

are primarily concerned with verbal behaviors in the cognitive domain. 

The subcat-egories subsumed under these headings 'have, to some extent, 

been ordiered according to their complevit^. 

C ategory I: "Gives hiforma t ion" 

Su bcategory ) ; Identity-Situatlun includes factual information 
regarding personal events ("I got a puppy for my birthday") anch labeling 
wilhokH further descriptive or differentiating detaile. 



This presentation is limited to the most salient points of defini- 
tion. A comprehensive listing of examples for each subcategory com- 
ponent is presented in the Observer's Manual. 



Subcategory 2; Prediction, Plan includes the child^a gues$ or 



hunch CMMl bet there's a frog in tehad* jar"). and projections of futui^e pl.i^ns" , 
('^Tomorrow Pin bringing all the stones I collected and going to start } 
labeling them for our science table"). j-i 



Subcat egory 3; Function, Process. . Instructions includes infoi:in4,V^ 
.tion regarding the functions of certain>tems ("The thermometer is 
how hot the water is"), how things work ("When you strike ttee key, it 
pushes the lever"), and how to^carry out a task or play a game ("The 
double checker can move either way"), 

/^^ ■ 
S ubcategory 4:/Differentiating Properties covers a wide range of . ; 

descriptive statemei^ts regarding sensory ^alities such as color and te>^- 
ture and defining Characteristics, e.g., size, form, /or quantity (i'There 
are 10 fish in the tank")/ v . 

'Subcatef^o ry 5: Rel a tions hips deals with comparisons ("It's the :\ 
same color as the moss") as well as with temporal, spatial, or ordlttal 
relations ("There are 60 seconds in a minute"). 

Subcategory 6: Category, Class includes behaviors in which 
groii^ membership is identifield ("flold is a kind of plant"). 

Sub category 7: Causal Re asoning, Pro bl em Solving includes an 
attempt to explain why things happen the way they do ("It* s heayier than 
water; that's why it sinks"), and a solution to an identified proMem 
("What we need are some railings so the cars won't go off")* \ ^ 

Cate gory II: "Asks Questions" ' . 

The , subcategories in Category II are the interrogative parallels ol 
thosve described above for Category I. For example, if the child points i 
to an object asking "What's that^ " it would be entered in Category U, ^ 
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Subcategory I, Identity- Situation ; the question "Which is bigger?" would 
be entered in Subcategory 5, Relationships * Subcategory 7, Causal Rea - 
jsonlng; Probiem Solving includes the child's inquiries regarding the urider^ 
lying c^u^e of natural phenomena ("What jmakes the lightning?") or an 
observed event {"Why do some things stay on top Of the water and not 
others?^'). 



Category III: *^Expresses " 

Category III includes both verbal and nonverbal behaviors that are 
primarily affective in content, i.e. , expressions^ of feelings and attitudes 
(negative as well as positive) and of preferences^ and needs. 

Subcategory I; Routine Needs refers to4behaviors such as pencil 
sharpening or taking a drink of water. » * 

gubcategory 2; Needs - Social /Physical includes complaints 
("Your paper is taking* too much room"; "My knee hurts where I bumped 
it"), and requests ("Move over so I can see"). 

Subcategory 3; Needs - Task Related includes general' requests Q 
(or assistance, materials, or equipment ("We need stuff for a collage*'), 
and re<)uests for approval or recognition of one's own work. 

Subcgi^fggory 4; Preferences and Desires includes responses 
given within the^^context of explicit choice as^well as more general expres- 
sions of individual preferences ("I hope we have tuna for lunch"). 

Subcategory 5: Feelings, Attitudes, Opinions includes negative ♦ 
as well as positive expression ("I hate cleaning up"), and beliefs ("Girls 
are smarter than boys"). 

Subcategory 6; Interest in Ano th Q.t:*s Attitudes or OpinionB includes 
quesUons such as "\Vhat's your favorite progrAm'*" and "Do you like to 
go to gym'' " 
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Subcategory 7; Affection, Warmth, Humor includes nonveirbal 
b*^havlors (child strokes rabbit gently) as well as verbal statemerit^ xMi*)^^ 
eating J positive relationship with another ("We're best friends, tight? "J^- 
good-natured joint laughter, and sharing ''joke'' with 'pleasure evidenced 
by all concerned. 

Su bcategory 8;. Concern for Others includes nonverbal behaviOitj,§ 
showing willingness to help or share as well as verbal statements or qti^^r:s 
tions indicating support of another's efforts, or considerajtioA for anothiDHg: 
feelings and well-being ("You can have some of mine"). . 

Siibcate^o r.y 9; Unwillingness to Help or Sha re, incl^des negative . 
responses to requests for help ("Go get it ydurse'i") or for sharin^belon|- 
Ings* materials, or equipment ("Don't give them any; they're just for 
us"j. 

Category IV; "Behaves D es tructively" 

Category IV ig rigorously defined to refer to behaviors in which • 
there is direct evidence of physical abuse or threat of physical abuse. 

S ubcategory 1: Vc?rbal - Initiates includes threats of physical 
abuse as well as extreme taunting, derisive behavior ( 'You're the 
stupidcHi one in the class; you never know any of the answers, dumb- 
dumb"). 

Su bcatef ) ory 2; Verbal Defens e refers to retaliatory threats ("If 
you do that, I'll punch you right back"). 

Subcategory 3; Phsfsical - Initiates is limited to those overt behav- 
iorn that appear intended to cause injury. 

Subcategory 4: Physical Defense is limited to retaliatory behav- 
iors that appear intended to cause injury. 
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Su bcatMQyy 5; Takes O ther^s Belongings is limited to overt b^h^V- 
tors that appear intended to be destructive, 

SMbec^tqgOry 6: Destr b ys Ano the r* s Work , same as Subcutegory 5, . 
* above. • ; * 

Subc|t fc OgOry 7; Abus e s Material and Equipmenti same as Subcate- 
•goty 5, above^ ^ 

S61;>ea|;egpry 8; Challenges Established Classroom Rul es includes 
negative responses by child to reminders regarding established clasB- 
.rooit) limits and expectations. 

Cate gory V; "Organises and Manages " 

■ . ^'^ ^^ / 

Category V represents the attempt to assess the extent of child 
. autonomy evidenced in the classroom. 

Sub category, i* Records Choice of Task (on bulletin or blackboard). 

Subcategory Z: Sugg ests Task or Act ivity ("Let's play with the 
blocks"). 

S ubcategory 3: initiates Task (Child goes to an easel and starts to 
paint). 

Subcategory 4; Commands, Directs refers to managerial rather 
than Instructional behaviors ("Get the blocks"). 

Subcategory 5: Initiates Atten tion al Focus refers to a situation 
' ♦>Jj(h^re the child enthusiastically calls attention to an event or feature that 
he bolt&ves is of general interest ("Hey, look at the plant; there's another 
green thing coming out"). It should be noted that calling attentio'n to 
one*s own work would be entered in Category III, Subcategory 3: Needs * 
Task Isolated , 
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Subcatogo r y 6: Seeks Answers requires some evidence that a qije$ii;|^ 
tion h9S been raised and a possible resoiiJrce identified ("Let's look at fli6r | 
chart* -it tells you how much you need"). 

Subcateg^cyy 7: Selects Materials refers to the selection of mateft- 
als and equipment from the cajjinet or shelf where they are stored. 

Subcate^ixSj^ Replaces Materials , the child return$ material ahd 
equipment, as above. 

Subcategory 9: Straightens Up Work Area includes behaviors such ag| 
ifVoppinj^up spills or picking up puzzle pieces that have fallen onto the 
floor. "X, 

Subcategory lb: Attempt s to Re solve Conflict refers to an attempt j. 
by the child to reduce conflict by Clarifying a situation ("We didn't mean 
to knock \ over; It was an accident"). or an attempt to settle a dispute hy^ 
compromlscK ("Since they won't let us lyiake it longer over there, wiei'U 



leeV 



make it longer over here"). . *' • 

s. 

Catego ry VI: "Represents a nd Symbolizes" 

Category VI focuses on the child's aesthetic and imaginative expres*- 
sions as well as symbolic interactions. 

Subcategory 1: Str u ctures Dramatic Bpisode includes behaviors 
relating to dramatic play in which the^child may assign roles and give 
directions ("Let's pretend this is the bus and I'll be the driver"). 

gnhrahn^/nry 2,i Elabora tes Dramatic Episode includes role-playing 
and use of objects to represent other things. 

Subca tegory 3t Makes Descr iptive C omparison s includes the use 
of figures of speech or idiosyncratic modes of description ("I feel cold 
like a butterfly"--shivering child). 
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be a t e g o try 4 ; Imp r o vl s e s^ Sou nd refers to play with syllablesA ini- 

Hal c6n$onanj;$, words, or tapped out rhythms, etc. 

I 

Subcategory Si Makes up Story, Song, Poem includes child's cr^a- 
live e.5cpressions whether in the form of story, song, poem, or dance, 

guOca t egpry 6: Tells Familiar ^tory may also include the sharing 

of a riddle br familiar song, or part ot a TV show. 

\ 

S^abcategory 7; Narrates Sequenc^ ol Eve nts refers to both personal 

1 ■ 

and historical events. i 

• i ^ . 

Subcategory 8; Shares Symbolic Experience refers to sharing pleas- 
ure in a book or picture with emphasis on the content. 

Subcfifcegory 9t Decodes/Pronounces (for practice) is limited to a 
mor'e routine type of reading out loud to another as part of skill practice 
with emphasis on decoding practice. 

A single DCB Form is used for each five -minute interval of obser- 
vational recording with a total of. 12 DCB Forms used for one day of 
observation. The observations follow a systematic course that is designed 
to provide 'representative samples of the behaviors of all the children in 
the classroom as they are observed in groups of various sizes and par- 
ticipating in ongoing activities with and without adult intervention, Activi- 
grouping, and adult role are indicated for eaqh DCB Form, 

At the end of each five-minute observation period, the observer 
completes a brief rating scale indicating the extent tb which coded entries 
were activity-related and the degree of task persistence encountered. 

The CI as s r pom Sca n provides a measure of the behavior of each 
child in the classroom during each of six time samples during the day 
(i.e., whether involved in an activity, observing, involved in a social- 
physical interaction, destructive act, or shov/ing "no observable focus*'), 

13 

ID 



It also provides for a description of the number and kinds of all ottgoing 
activities and groupings, identifying adult role, if any (i.e., suppottlng. 
ov directing). Hn relation to each group. The activities are futtfher 
difscribcd as to the perceptual modes involved, degree of abstraction, 
and dimensionality. Each scan is administered immediately utmr ^ 
of two DCD Forms has been completed with a total of six scans fbr Oa^ti I 
day of observation. 

Overvi ew of Previous Findings 

Preliminary work with the DCB was based on three grotips of elc^ . 
mentary school children age five through eight. The first group was 
drawn from the Bank Street School for Children, which has an open- 
classroom approach, and consisted primarily of middle-class chil|irem 
The scconr' group came from the BAnk Street Follow Through classes In 
inner *city public schools- -also with an open-classroom approach. The 
third group of children attended non- Follow Through inner-city public 
schools with a traditional classroom approach. Thus, three distinct 
reference points were provided for examining the DCB data. ^ 

) I ■ • ■ 

Results 

' ^The results arc summarized in Table 1. Substantial differences 
wor^ found among the three groups from which data were drawn. The 
Bank Street School for Children classrooms totaled twice the number of 
Interactions found in the traditional public school classes, the number of 
interactions in the Follow Through group was closer to. the number in 
Bank Street School rather than in the non^Follow Through group, showing 
6(i| percent more entries than the traditional classes (p < .01). The total 
nuV-nbcr of children's Interactions recorded in each setting provided on6 
means to describe a classroom environment along a passive-active 
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1 • 
continuum. In.thqsc resUUs, the' Bank Street School for Children repre* 
bqMs one polarUy--"very active"--and the traditional publife schObi 
classes represent another--"least active" or "passive'^.-with Ih'^ FoUdW 3 
TRrough classes notably more active than passive; the traditional pubHo. * 
school classes were more passive. 

ft should be emphasixed that although this analysis is limited to a 
measure of interactional behaviors, there is no assumption that nonintfefi -^l 



actional behaviors, e.g. , the child reading or writin|[ by hims<il£| are 
either passive or nonproductive. However, si^nce observers sought to 
record children's behaviors in those situations that fostered intxjractiOni I 
the results would appear to reflect the general level of interaction f0s$i- i;? 
bie within a given classroom. We may conclude-from the sharp differ- 
ences In the total scores that the children in the traditional clas'ses wo?re\ 
either far less active, or were engaged in tasks that provided less ojipor- 
tunlty for interactions. * 

In all three types of classrooms; the most frequent interaction was 
that described as giving information, i.e.. Category I: Cognitive Domai||# M 
The traditional public school classes showed. by far the fewest of the . 1 
higher-order cognitive interactions. The Bank Street Follow Through 
Program had more than twice the number of higher-order cognitive inter* 
actions (p < .01) and the Bank Street School for Children had almost three 

o . 

times as many (p . 00.1 ). 

The second cognitive category, that concerned with asking questions, 
occurred much less frequently. When examined on a proportional basiS|. ^ 
dLf(erences among school groups were even more massive than those ^ 
found with regard lo giving information. The children in the Bank Street 
Follow Through classes asked more than foUr times as many questions 
as the children Ln the traditional public school classes, and thjt Bank 
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Street School children were found to ask questions mofc than five fcii^^s 

m often as these in Follow Through and the traditional public school 

(|>< . Oi, p < iOOl, respectively). 
I 

The second most frequent category of behavior recorded by our 
system was that concerned with, the expressiveness of children (Category 
III). The Bank Strept Follow Through groups and the Bank Street SdhOOl 
classes showed approximately equal amounts of expressive behavior, but 
both ftequehdes were almost twice that found in the traditional public 
school Classens (p < . 04, p < . 01 . respectively). When examined on a ,^ 
^pjoporttonal basis, differences among the three school groups in ox^res^ 
iiVe Interacfions were relatively small,, especially when compared to the 
large dlPfer^nces found Sfmong the school groups in cognitive interactions. 

4l> ^'^^^^'^ interactive category that yielded large differences among 
the school groups was that concerned with autonomous behavior (Cate- 
gory V). Fy far the least amount of autonomous behavior was shown by 
the traditional public school groups. Those enrolled in the Bank Street ^ 
Follow Thiough Program showed almost three times as many such behav- 
iors, wbilt the Bank Street School children showed more than twice as 
many autonomous behaviors than the Bank Street Follow Through children 
and more than six times as many autonomous behaviors than the traditional 
public sciool children (p < .Ol), 

Anong our most surprising findings was the low incidence of destruc- 
tive bchivior found in all three groups.^ The %gest difference was be- 
tween tie Bank Street School and the two public school groups (p < . 01 ); 



^Tle incidence of such behaviors is reflected by the following mean 
total fremencies for the three groups: Bank Street i>chool - 1.9, Follow 
'Phroughs 10, and Comparison = 10« 
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there woro five times the number of destructive interactions in b6th ftl^ 
Bank Street Follow ThrougR classes and in the traditional public ScHfjol 
classes* However, when these findings aire considered in relatt6jri tO the 
prOf>ortlOn of total behaviors represented by ^ach group's score, the num^ 
her of destructive acts In Follow Through actually represonti a contldeta- 
biy smAller proportion of Its total behaviors than does the traditioYial 
group's similar score. 

' JMlhough the number o£- adult-elicited bqhaviors was similar tit the 
three groups, there was a far greater number of child-initlat^ behaviors 
In the Follow Through and Bank Street School groups than in tie tradi- 
ttonnl classes (p <. OOl). However, those in the Bank Street g^l^ool 
showed more child-initiated^ child-to-adult behaviors than did tiose 
eiiher in f low Through or the traditional public school classis* 

in sum, our preliminary data indicate that the DCB shoW4 ptomise 
of contributing to a body of information that will ultimately describe the 
'^ecology of the classroom. In spite of the dramatic differences 'ii class- 
room management and teaching styles in our sample, a number of uni- ^ 
''form patterns of classroom interaction were observed. The m^cst fre- 
quent category of classroom interaction, in all cases, was that df giving 
information. This form of cognitive interaction occurred most o^ten. 
According to our method of categorizing classroom interaction, tie 
second most frequent for^^^of interaction was that concerned wlthexpres- 
slve'behavior. In all of the schools we studied, destructive behavor was 
found to occur relatively Infrequently among the children's intcraclons. 
The most striking differences found among the groups were in thc:otal 
amount of Interaction observed, the amount of conceptually-basedinfor- 
mation exchanged, the amount of' questioning and autonomous behas^iori 
and the amount of child - initiated behavior. 



/ 



18 



24 



/ 

/ 

/ 



Although the Follow Through classes were more like the tradUional 
ptsbUc School classes than the Hank Street School In tei^ms of the SES 
Backgrounds and school settinjis, {he DCB findings indicate that the chil- 
dren's interactions in Bank Street Follow Through classes were niore 
itko those in'the Bank Street School classes than' those of the tradltionul 
public achool. should be pointed out thai the Follow Through classes 
.did not all manifest identical patterns, but could be identified along a con- 
^ linuum, with some classes showing patterns very similar to those of tke 
Bank Street School and isomc showing patterns closer to those of the tra- 
ditional pubtlLc school. > 

i ' ^ ^ 

tt should also be noted that the DCB records and describes what 
the observer focuses upon, and thereby captures the quality of interaction 
that occufd, but does not report the number of activities that arc taking 
place at the sa^e time.. Comprehensive coverage of classroom transac*^ 
tlons is provided by supplementary observations using the Classroom * 
S^an, which yijelds a detailed description of all activity*grouplngs occvtr* 
ring at six time periods throughout the day. 

Cur rerft vyork - 

Current Work on the DCB is coticernod with refining the coding 
procedures and expanding their coverage; cross- validating the previous 
flndings^with a new set of equally varied classrooms and extending the 
comparative study of classrooms to subcatej<ory descriptions as well; 
examining the path of change in the data recorded by the DCB during the 
course the apadcmlc year (patterns of classroom interaction obtained 
from DCps in the fall will be comparojj^ovlth those revealed in the spring); 
determining the degree of relationship 'between DCB scores and an assess 
ment of teacher -behavior* (an independent assessment ^pf teaching behavlbr 
will be related to classroom interaction data obtained trom the DCB); and 




assessing th^ reliability of DCD scores by romparing data from two dif- 
ferent observers who observe in the same claasroom during^ the safirli^ 
time period, witT» data based on observations by the same observetr on 
different days. Although previous work with the DCB indicates that it is 
suificlently reliable to differentiate sharply among different types of * 
ciassroon%(S, it is essential to estimate the magnitude of error attributable 
to variation among different observers and to day-tQ-day variaticin within 
tho ^arne classroom and the same observer. ^ \ " 

The foregoing dij^cussion l\as described in summary fashion.one 
effort to aTFrtve^at a m«*thod for observ^ing and recording childrer^'s IhFer- 
action In the classroom. DCB data may serve either as the hidependent 
variable or the dependent variable in an educational experiment because 
they describe phenomena that mediate between the educational stimulus 
and Us internalized impact upon children. The DCB* data serve as b^oth 
an index of the quality of educat^bnal intervention experienced by a class- 
r'oom of children and as an indicator of the kind of influence a particular 
f<^rm of educational intervention is likely to have upon its participants* 
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' OBSERVATION AS A TOOL FOR EVALUATION 
OF IMPLEMENTATION^ 



Gaea Levnhardt 

Learning Research and Development Center 
'University of Pittsburgh 

■ k 

_In the United States in the past 15 years the re^ has been a ijjarked . 
effort to^improve education at all levels with special emphasis at the 
jrrcschoot and elementary grades. This effort has resulted iti a pro- 
liferation of new subje^ matter sequences as well as more global edu- 
cational alternatives such as the "open classroom;'' The question now 
'being raised is whether or not the programs have been succ^ssfitll that 
is* requests are being made to evaluate the effects of ^^ucational ipno- 
vatinn. The form which the evaluations take has been largely a function 
of the nature,of the innovations.' Innovations which focus on the ptdcess 
of education tend to emphasis'.e classroom descriptions of attitude, cli- 
mate, and interaction patterns; while innovations which focus on aca- 
ddmic in-iprovcment tend to emphasize positive changes in stalidardized 
subject matter tests. 

His paper presents information which can broaden the intetpre- 
tatioh and utility of outcome measures on standardized tests. The 



^The research reported herein was supported in part by ^ grant 
from the Uitited States Office of Education to the Learning Research aftd 
Development Center. The opinions expressed do not necessarily^^fleet 
the position or policy of the Office of Education and no official eftdotse^ 
merft should be inferred. 
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purpose of this paper is to demonstrate the need for and means of 
ovatuating Implementation of educational innovations. It presents infor^ 
matlon which can broaden the interpretation and use 6f outcome measures 
on standardised tests. Measures of implementation can both clarify the • 
natures of the educational process and demonstrate the relationship of 
that process to observed achievement. Several assumptions are made. 
First, educational innovations need to be evaluated not only for the* 
obvious economic reasons^ but also in order to provide clearer inkight 
into areas which need improvement. Second, the reporting of educational 
outcomes without relating the outcomes to the Innovative process does 
not cOhstitjLite a meaningful evaluation. Third, an appropriate way t6 
evaluate educational innovations is to measure both input or antecedent 
variables and process or transaction variables, and to use those measut*^s 
to explain or predict outcomes (Stvifflebeam, 1971; Stake, 1967; and 
Coofly, 1971). The ma.in body of this paper is concerned with indicating 
what ty^U' of. information is useful for measuring implementation variabie$ 
and how to assure that those measures will be credible. 

Sotting 

The Learning Research and Development Center (LRDC) is cur- 
rently involved in evaluating its educational program in the Follow 
Through Schools, LRDC is one of ZZ sponsors^in the nationwide Follow 
Through program. The Learning Research and Development Center's 
Instructional Model (see Appendix A) is prersnt in seven Follow Through 
sites in kindergarten through third grades; eacli site consists of from 
two to seven schools. The evaluation effort described here focuses on the 
secand grade classrooms at four established sltes» those sites which have 
had tl>e program at least one year. T^he input data, which described tho 
entering aptitude of students, consist of the l^orge-Thorndike Cognitive 
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Abilities Test, a ^^enoral abilities test. The output data consist. of moa|i« 
ures on tho Wide Uange Achievement Te^t (WRAT). The data for Imple- 
mentation consist of descriptions and measures of the various dimoD$iOhS 
of the classroom obtained fi^om an instrument which was specifically 
deslgncj.cl for that purpose. 

All investigations which take place in a natural se^tting hav<3 soma ^ 
unique- restrictions and advantages associated with them. The advaii- 
tages art- the tremendously increased credibility and generallzablliiy oi . 
the information obtained. Clearly, if one can demori^trate that a pfo- 
j^ram can be ipiplemented and that the implementation improves per for rlri- ♦ 
ancc in such widely differing settings as Follow Through, one has builf 
a very strong case for thp program. The disadvantages, howeverj are 
also very great; they focus on the following three areas: the geographic 
location of the classrooms (they are widely dispersecf nationally )i the 
staffing at each site (it varies'in terms of the availability and willing- 
ness of its members to engage in evaluation activities); and thy record 
keeping process (no permanent or consistent record^ of testing and 
pijesrripHon are normally kept), 

' ^' ^^^'^^^^^^P'^^^ ^^ Q ^ J'^p jgQTP P^^^^Q " In strumen t 

Figure I shows the sequence by which an instrument for measuring 
the implementation of the l.*RDC instructional Program was developed 
and tricMl t»ut. f view the steps a^s 'OOcessa ry and sufficient for thd develop 
ment of an implementation instrument; however, 1 do not view thi^ as a 
unique solution to the problem of such development. The diagram i^s 
read in the traditional manner and will not be discussed in detail, bat 
It will be referred to throughout the paper. 
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The first four steps in the figure generated specific inforthaliOn 
about the LRDC program and about the underlying theories of education 
on winch the model was built. The discussions with developers and 
impld^entdrs brought out the concern that while thf» measures might be & ^ 
rcstricliftf to the unique aspects of LRDC's program, the domains of the 
varlsibles tapped should be generalizable to a variety of educational set- • 
tings. Staled somewhat diffcrenlly. the specific measures are nested 
within variables (potentially measurable in diverse ways) which in tufn [jM 
arc nested within fundamental domains of concern. The following list of ■% 
the variables that need tu be measured emerged from observing class-* * • ni 
roonis, discussing the program with developers and implementors, and ;| 
examining the literature: the context variables of each classroom, the 
alU>cation oTtime, the allocation of space, the assignment and measure- ]^ 
ment procedures, c la>ssroom management, and student independGnce. 
Measures of these variables should serve the following functions: (1) 
provide descriptive information about the field sites using the LRDC pro- 
gram; (2) provide a basis for comparing the laboratory and field schools; 
(5) provide a basis for comparing the tnodel and the field;, and (4) pro- ^ 
vi<le an explanation of output variance not accounted for by input measures. 

The Inxtruinenl 

After six major revisions of trial instruments, a field reading , 
version \ms developed. That is* steps 1-15 on Figure 1 were cycled *' 
(hrouj»h approximately six times before an instrument which could be 
used in the field was developed, Following the field test of this instru- 
ment, a final version was developed incorporating minor revisions which 

from the feedback of the field tester (Step 191, The instrument 
ilrteir. the nature of instruction, content validation, and the ^raining pro* 
gram for its use are presented elsewhere (Leinh^rdt, 1972). The 
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ijl3i;rumont covers four rlUjor areas: background, proscription, to^ting^ 
and t«^achoj; interactions, which will each be discussed briefly. 

BftCkg^rQund. The first four questions on the instrument proWdo 
background information on the classroom; the number of boys andglrjs 
involvodt the number of children who are presented on Ihe day of obsc^r- 
vatiioni thi? number of years the teacher has been using PEP-IPI; the age 
range of the class; Ihc si^ic of the class in square feet (transformed into 
^ the number of square feet per child), and the allocation of time and 
space for cKploratory, (See Appendix C. ) 

a j^TOSC ription. The next area on the instrument concerns j.rescrip^ 
lion informtition (see Appendix C). Question ^a on the instrument a$ks the 
observ<?r to list all of the IP\ Math or Quantification assignments written r 
on each child's prescription sheet on the day of the observation. The 
Information is obtained by looking at each child's ticket (sheet) or folder 
and recording the must recent list of assignments. Tlie question was 
coded in a manner which would yield information about the uniqueness of 
the list of assignments obtained. A ratio of unique assignmenfs (dif- 
ferent by units and levels) over total assignment was formed, giving a 
single measure of uniqueness for the classroom. 

To sting. The next domain is testing. To gather some Information 
about the procedures being used, question 5b asks the individual to list 
the dates of the last time a child was tested in any way in math (IPi or 
^*EP), Foi/'each date listed, the number of Hchool days from that date to 
the date of observation is counted, coding the date of observation as xero 
and skipping weekends. To gain infor ..lation about the time between testings* 
the average number of days between testings were obtained from the same 
prescription sheets used in or from the teacher's testing data im 
Quantification. Testing is bcint; viewed here as an opportunity for the 
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teacher to use information about the students^ performance fcO ^Um the ^ 
assignments given to them, /| 

\ • ■ ^3 

The major function of testing is to accurately place arid then itiOni- 
tor the progress of the child in the curriculum; From the testing infor- ^ ^ 
mation, prescriptions for the daUy activities of the child are Written 
which in turn affect the rate of progress through the currK^ulum (aias^r* , j 
1968). One of the aspects of testing which is assumed to be impo^^^tit 
for an individualizied program is that it occurs at regular enough inter- 
vals to provide accurate information about the changing curriiculaf ne^ds 
* ot the child. Unfortunately, regular intervals are not defined anywhere ^ 
in the titer a tare about implementing the LRDC program. However, from 
discussing the problem with program developers, there was a consensul' 
that a reasonable time l^jice between tests ranged from five to ten days; 
> a o^ne-wcek lapse Svas* reasonable while a three -week lapse was unrea- 

^ ■ ' / ' 

sonable* 

* Teacher interactions. In any Individualized program, a great 

deal of the success or failure is dependent upon the teacher's inter 
actions with the students. The teache * is the one who sets the tp^ne or 
atmosphere of Indlvidualixation. If his/her actions convey a^rf^nse of 
•group rather than Individual treatmejjt, ^then the effort to i^ndividualize ' 
will have failed In a fundamental way. Queiition 6 attempts to tap some 
of thi? relevant information about teacher and student interactions* The 
leacher's and aide's Intcractibns were observed dufing the time of day 
when prescription work was done. Measures obtained from the sixth 
question were: (1) the frequency of contact, (2) the content of the con- 
tact (Le. , what Information Is being transmitted from the teacher to the 
child In terms of Intellectual and affective communications)! and (3) the 
distribution of contacts or the way in wfiich the teacher allocates her 

^attention among the students. 
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Distribution, frequency, and content of the teacher's contacts 
are measured by observing the teacher for 15 or 20 minutes while (s)he 
Is truveting. During the observation, every separate interaction with 
a child Is coded and placed in one of «ix celU on a sheet, depending on 
whei'e the child to whom the interaction was directed was localcd (see 
AppendiK C). The tells correspond to areas of the classroom; the num-y 
ber of children located in each cell is recorded. From this system of 
observation, one can obtain measures of frequency and distribution over 
the class of coniacis made. The content of the leadu'r's stiitement was 
coded as: general positive, general negative, management, positive 
management, negative management, cognitive or subject matter oi'lohtcd, 
positive cogn,tiv(^. negative cognitive, managenu-n! cognitive, positive or 
negative management tognitive (Lelnhardt, 1972)» While these categories 
do not cover ail of the nuances of a teacher's l^eluivior, they arc the 
minimal ones that describe behaviors important to children's learning 
{Reynolds, Light. Mueller, 1971), 

The coding of the observation section is complex: thereforCi the 
measure will be underlined, followed by a brief description of the coding 
procedures* Total frequency was coded by counting tlic total number of 
contacts made for the entire observation time (IS to 20 minutes), and 
includes: management, cognitive, checkoff, cognitive Tmanaficmcnt, 
unatt(tched positive or negatives, and any uncodcable X*s. The fr equine y 
for cognitive 3 was obtained by counting all cognitives plus cognitive 
managements for the entire observation period. The freqt^oncy of manage- 
m ents was obtained by counting all other eoniatts made. Thus, frequency 
of management plus frequency of cognitives equals tota! Sequency* The 
percentage of negative contacts was obtained by adding ail negatively 
coded contacts (negatives, negative management, negative cognitive, 
negative cognitive-management) and dividing that by the total number of 
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contacts made. (The perccniage of positive contacts was SO imall ^fi^ 
unvaried that it was not coded after the field trial was examined* ) 

. There arc three distribution measures: total, management* s^nd 

cognitive. They do jiot add up to total distribution. The System for 

coding the total di stributi on will be explained to provide an example*^ the 

other two codes are similar. A distribution was calculated for each eoll 
Z 

by the formula ( 0-E) 
E 



Whe re ; 



The observed total number of contacts fo.r* 
that cell (or observed managements or cog- 
niti ves ), 

^co \ ' The expected total number of 
T . / contacts for that cell. 



T Total contacts made over all cells (or total 

^° cognitives, etc) 

T - Total number of children in the system /' 
' obtained by 6 

Z(Ch) 

_ . ' / 

Ch = The average number of children per cell 

based on 2 counts or 4 counts depending on 
the two forms, 

■ . / . 

The distribution measure is then obtained by summing over all cells. If 
the observed frequency equalled the expected frequency in each cell, the 
total score would be zero. Thus, the smaller the measure, the more 
evenly the leachor and aide are distributing their attention. 
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The throe variables^ -frequency, content, and distribution- *are 
each Important Indicators of the leachor*s style. of Interaction^ The 
(roquoncy of cdnt»*^l Is a good moasi^ire of t/lie "travel'* rate in the room. 
It is also a reasonable indicator of how lonj[j children wait before they 
are able to get the teacher's attention. The major difficulty for obser- 
viatlon of tho content of a teachc r*s remarks lies with the decision of 
which aspects are most relevant, I have chosen to focus on a rather 
simple distinction between negative statements and all others, and 
cognitive versus imnagcmenl statements. The collapsing of the existing 
categories was done for several reasons: first, to increase inter- 
observer agreement; second, to increase the frequency of the observa- 
tion of the* categories within the Umlted time of observation, and finallyi 
to focus on the most relevant parts of the teacher's speech, Agajrt, thok'e 
Is the problem of incomplete rather than inappropriate measures. Those 
parts of a teacher's interaction which would seem most relevant to the 
student's advancement are those which concern the affective dimension 
and subject matter content of the communications 

' Other Measures Obtained, Several other measures ^ere obtained 

■ i.,„.r„ ., „- | . , --r, , », ^- 

which have not been included in this general discussion/ Some w^re 

2 

obtaiiied by directly <iuestioriing Educational Specialists after the Instru- 
ment had been administered, others we re. obtained from the instrument 

» 

Itself. Additional measures include: the number of adults observed 
traveling; whether or not the children get their own work (information 
was obtained by inie reviewing the child- -it provided useful anecdotal infor- 
matidn,'but was no( very generhlixable at fhe classroom level); hours 



f An Educational SpecJalist is a person at the Follow Through site 
who is responsible for Implementing and jpupporting acauemic and parent 
involvement programs. There is usually one specialist for every elghi or 
nine tcachefa, 
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assignee! m math and reading (regardless of whether the reading is an. 

l,RDC prouiam) which ;vvas obtained fr^m the site specialistSJ and nn^Uy/ 

the numbe r of days the teacher was absent during the year, aldo obtained 

fi*om Ihc specialists. Those are measures of implementation which 

r • 

might; be relevant to the student learning outcomes. 

Measures Omitted, Several of the original measures In tho 
inBirumcnt were not Used because of a desire to pare down the number 
of variables being examined tu those which seemed to give reliable, 
useful, and inte rpretablc results. Th.e questions either condensed or 
cj.milted ccuuernrd exploratory (except those already mentiontid), the ^ 
room map with teacher tra^vcling pattern, the percentage of» contacts ^ 
which were chjid initialed, and the child's interview* The room map was ^ | 
very useful in interpreting and in some cases correcting tho information 
recorded, during the teacher observation. 

The one major observational measure which was lost was tlVe per- 
centage of lontatJs which were child initiated, betrause good 'agreement ^ 
tabove uO percent) was not obtained with observers. A sourcO of the low 
agreement is the lack of a .consfstent signaling system acrbss classrooms. 
In most c lasses, from two to four different signals were used: hand 
raising, flag raising, finished work turned over on a desk, hands folde<f 
or a bakery typo number system. Hand, and to some extent, flag raising 
Are fairty easy for the observer to sec and record, but it is hard to re co I'd 
both together. Other kinds of Signals used are almost impossible to 
observe ai<uraiely if they occur in conjunction with another system. 

l-*roccduro 

, ^ c 

The instrument was administered twice id all second grade class- 
rooms in the four established sites. One classroom at each site was 
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♦ randomly chosen to be observed twice for cacH adnilnisl ralion, The 
Rrsl: admlntstraUon, the field test, \va» done over a one-month po^riod 
,by one obsci-ver. The second admihisl ration, the final pass, was done 
by eight observers (two for each site) over a two-week period. 

In accordance with step 20 of Fij^ure I, a trai'^inji-pt'og-tft-ni to - -s^ 
teach the adaiinislratora to use the instrument was developed (heinhoidf, 
19?2)s The program consists of training for knowledge of the inshnicMons 
abllUy to unitize, and ability to catcgori/.e teachers* verbal inleractiohs. 
In -class training focused on reri);'din)ij| the distribution of teachers- inter- 
actions and record iolleclion. 

By the ctniipletion of the sei tjnd pass, data had been obtained on 
30 class rooms. . On twiMUy-one ilassrooms there were two «pls of 
measures and on eight there were four sets. This provided enough data 
to estimate the reliability of the instrument. The next si-ifion exdLmines 
the problem of observational reliability, followed by djst\ts«uintf tif the 
inter-observe r reliability, short-term teacher stability, and long-trrnx 
reliability of the Instrument, 

Reliability 

The concepts of reliabi li(y and validity involve procedun^s by 

which confidence in a measuring deviie uiay be established. Phcy lt»nd 

support to the assertion that the liieasure consistei^ly reports the same 

« 

situation the same way, and that the measure actually repr<'sentH that 
which It is supposed to be measurinj^. In our l ase* the major i liallonges 
to reliability were that cli ffif rent observers regarded tlie sanie evvnl dif- 
ferently, and the laik ol stability or representativeness of the b(»hyvi<>r 
observed- The domains of jxirtiiular nitt*rest here are the inte r • rtbse rve r 
l*el iabilitvy and the stability of temher behaviors over time. 
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Sex^eral procedures are available for'calculatinft either ih€ overall 
rellabUily of data or some specific aspect of reliability, such a3 intor- 
observer ag-recment. No one of them Is completel/appropr late to th6 
^problern of calculating '^^^ reliability of our data. In keeping with the 
Sfitrit, If not the sf^ecifl^ method, of Cronbach's ^'Theory of GeneraU^a- 
bnUy»| A LlbcraUiation oV Reliability flieory" (Cronbach/ Rajaratnam, k 
Ciie&er' lg63) where 'Reliability Theory' is interpreted as a theory 
regarding the adequacy y/ith ^H^hich one can gei;icralixe from one observa- ' 
lion to a universe of observations" (p. 137), I will present a variety of 
evidence, some of which used traditional estimations of reliability coef- 
ficients and others did not, to support the gencralizability, or lack of it, 
for this data set- 

* 4 

Three aspec ts of reliability were estimated: inte r -obse rve r . « 
sKorl-lerni st*abiiity of the leaclicr behavior, and lon^-term reliability. 
The results arc reported in Tables 1,2. 3. and 4. Thv reasons for 
keeking more complete information about the reliabiiity of the instiument 
go bat-k to the initial points mentioned ih Uiis papcr--there is a very 
definite need that the daia which represent measures of implementation 
be credible. One way of establishing such rredibility is to show that the 
data arc reliable and valid. 

[:ntc)>-Qbservcr Reiiabiiit'y . Whenever human observation Is a 
basics for measurement, one is fated with the problem of individual dif- 
ferences in observers producing differing results, wht»n m fact, they 
should have produced the same results. If one in dealing with several 
observers, the problem is to get .\ll tht- observers to cede the^aijic event 
in the same way. Tabic I presents the" in-class inte r- obse rve r agree- 
ment for nine observers. Agreement was checked by taking a ratio for 
ofTctv lalogory between two recordings of one situation at one time. All of 
the observers had an agreement check with me, and in those cases where 
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Developer & 
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Developer & 
Observer G 



Observers 
F &'G 



Developer^ 
Observer 



two people were trained together, the ratio of agreement between them is 
also given. This tablt? presents the rai.ge over individuals and categories, 
and how mucft each individual agrees on each category. The overall 
reliability for observers is 82 percc^nt. The range across observers is 
66 to 100 percent and across categories is 29 to 98. percent. Because 
the category distribution ofsmanagement contacts had such a low reliability. 
It was dropped. 

Shprt'te rm<Stability . Short-term stability refers to the stability 
of a leacher^s observed behavior* ^ve r approximately 48 hours. This was 
checked by having one randomly selected classrooifi at each site obfid'rVed 
twibe in two days! The reason for calculating short-term stability is to 
show that the observed teacher characteristics remain relatively stable 
over a short periqd of time. 

A summary of shoft-term.stability estimates for several variables 

\ 

is given in Tabic 2. The variables chosen were ones which would vary 
over a 48-houV period (or less). The observers were not instructed to 
rccounttenrollmont, rechcck the number of years of expe rience , or 
rocordWher context data. Therefore, these variables are not included 
in the estimations. 

The estimates given in Table 2 were obtained from a two-way 

mixed-model (rows (teachers) random, columns [time] fixed) repeated 

measures ANOVA of tec^chers bv time for each category for each pass. 

(That is, separate ANOVA's were calculated for each pass and eacli 

category. ) From this, two estimates can be calculated which account for 

the variance due to toachers. One is Cronbach's estimate (which is not 

presented), (Cronba.h, 1971); the other is an eta squared. The eta is 

obtained by dividing the sums of squares due to teachers by the total sums 
2 

of squares (i.e. , n SSTeache rs /SSTotal ). This estimate gives the 
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TABLE 2 

Estimates Qf Short-Term Stability of Teacher Behaviors 











Variable 


1st Pass 


2nd Pass 


i 
:l 


?ercerjt of children 
present 

! 


.80 


78 


i 

■ .'u 

1 


Numtjer of days since 
the l^t test 


.99 


M 


« z 


Num^r of cognitive 
statemi|nts 


.82 


;87 


'j 

' ] 


\ « 

Number of management 

statements 

i 


.80 


.96 


■ 4 


j 

DIstrlbutibn of cognitive 
statements 

! ^ 


.83 


.066 


1 


Percent pf unique 
assignmcihts 


.50 


.86 


1 

■I 


Percent of negative 
' statements 

1 ' • 


.37 


.98 


\ 
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amount of variance explained by having the sams teacher versus different: 

teachers observed. Tabic 2 shows the results of estimating short-term 

reliability by an ^'ta srquarcd, (Both estimates vere very close; for 

further discussion of a comparison of the two estimates, see Leiiihardi, 

1^72.) The average stability for pass one is ,73, and .78 for pass two. 

« ■ 

The 'extremely low stability (, 065) associated \yith the distribution 
of cognitive contacts on the second pass is du^^ largely to one c\aB&t;oam* 
As shown in Table a classroom at Site l^Kad an extremely low/Qistri- 
bution score on iru first observation and ^n extremely high sc^re on 
the second. On the second^ observation/{)f this classroom, th^ teache r 
was planning a field trip for t!ie day b/\i kept the students V" school one 
extra hour so that the observation c/vild be made--a factr unknown to us 
at the time. Almost no c lassroon/^^work went on during the observational 
period; there was a high degree of disruptive behavior. Due to the low 
stability of the distribution of cognitives in the classroom for the second 
pass, the measure v/as substituted in the final data set by the measurers* 
from the first sot. 

i. 

Long-Term Reliability . Long-term reliability refers, in this 
case, to the consistency over one and one -half months of some of the 
measures which remained unchanged from the field test and final version. 
The purpose of estimating this is in part the same as estimating short-terfn 
stability. But it serves the additional function of estimating the reliability 
of the instrument in recording some events which presumably do not 
change. The reliability estimates are reported in Table 4. 

The stability of the following measures is reasonably high: teacher 
experience, enrollment, ratio of boys to girls, and sequence of explora- 
tory- -while the stability of the remaining measures is low. The con- 
seqiiencV of the low reliability is that the two variables will not be used 
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. TABLE 3 




• 




Distribution of Cognitive Contacts: 


Raw Scores for Doublechecks 








1st pass 




2nd pass 






Time 1 


l\.m 2 


Time 1 


Time 2| 


1 


5.33 / 


4.03 


2.43 ' 


20."^^ 


2 


8.06 


6.94 


4.04 


7.27 


3 


25.36 


15.14 


3.4 


4.7 


4 


6*08 


8.45 


7.7 


2.4 



f • ■ * ■ 

Note^ The classrooms for each site differ from the first pass to the 
second. 



TABLE 4 



Estimation of the LongTerm Reliability 



Variable 


'12 

* 


Years of experience 


.92 


Enrollment • • • 


.70 


Ratio of buys to girls 


.80 


Percent of children present 


.34 


Square feet per pupil 


.J3 


Sequence of exploratory t- 

9 
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to predict performance, although the attendance variable will be averaged* 
across times and used for descriptive purposes. , 

1 In addition to the concern about the long-term stabiltty of the 

variables, there is a concern about the type of information gained 
between the first pass atid^ the uecond, or the difference between sending 
either one or eight people to collect data. One expectation concerning 
the differetice between the groups would be a greater overall varlanco 
for each variable on the second pass than on the first, attributable to 
individual differences between raters. A test was made for the assump- 
tion of equality of oyerall dispersion. The variance-covarlance matrixes 
from the two passes were found to come from similar populations. That 
is, the null hypothesis H^: D^trD^^/i is retained {Fr.921). This is not. 
therefore, an estimate of thi reliability between one observer and eight 
observers, but rather, evidence for the assertion that the two situations 
vfi ve comparable in variance and covariance. . 

Valicft ^ty 

There are two challenges to the validity of this instrument. I^irst, 
it ij5 possible that the characteristics examined are measured accurately, 
but ar^e not the relevant ones in terms of the final performance of a clasd. 
Second, the characteristics selected may be the most important ones, 
but the manner of measuring them is not sensitive enough to reveal sig- 
nificant (in the sense of useful) differences. The first challenge has been 
tPisi ussed in the presentation .pf the instrument. The second challenge 
was answered in two steps. First, intercorrelations between the input, 
process, and output variables were examined to determine if the relation 
ships among the variables we re consistent with the theoretical basis of 
the model (they were). Then, the relationships between the output atid 
process variables were examined controlling for the input variables 
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(e. g, , a partial correlation) (Loinhardt, 1911). A eelected group of the , 
residual process variables (to be discussed later) accounted for 46 per- 
cent of the res/dual output variance (Cooley & Leinhardt. 1973), The 
instrument appears to be both sensitive to differences between class- 
rooms and useful in explaining outcomes in terms of achievement scores. 

Findings 

The purpose of developing the implementation instrument was to 
provide a source of information which could broaden the interpretation 
of achievement data. The specific information obtained from this can 
be examined in terms of three questions. First, which changes in 
variables in the classroom appear to affect the achievement of students? 
Second, what differences are observed between the implementation of the 
program in the field versus its implementation in laboratory classrooms?. . 
Third, how is the educational model transformed when it is implemented? 

In considering the relationship between classroom variables and 
achievement variables, it is crucial first to consider how initial input, 
process measures, and output measures relate to each other. It is 
obvious that one could obtain strong relahonships between what appeared 
to be measures of process and* output by merely having the process mea- 
sures be surrogate input measures. For example, teacher experience 
with our program correlates , 47 with student achievement, but it also 
correlates . 58 with IQ. When IQ is partialed out of both achievement and 
teacher experience, the partial correlation between expcrit»ntc and 
achievement goes to . 05. That is, in this case teacher experience is 
confounded with input variables. In this data set the IQ means (considered 
input) correlate .68 with arithmt tir means which leaves 54 percent of 
the variance in achievement to be explained by measures of the classroom 
which are not themselves accidental measures of input. 
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Table ^ shows the partial correlation between five classroom 

residuals and Wide Range Achievement Math Residuals. The variables 

which are negalivoly related to achievement when IQ is controlled are 

a large class, more boys than girls, and a larger number of negative 

contacts. Only Ihr last variable is one which relates speciCically to the 

LRDC program. That is, in the LRDC model, the emphoi^ is on having 

the teaclier reinforce learning behaviors rather than punish Vnappropvi?ite 

« 

behavior. The variables which are positively correlated with achieve- 
ment arc the nunibcr of days between tests and the amount of class lirhe 
spent on mathemalico. The sec»)nd finding is predi^-table; although it 
can influcnio decisions about amounts of time devoted to any one cur- 
riculum area. It does not significantly influence the model. The finding 
thai higher athicvoment is associated with greater time between tests is . 
both startling and iiuriguing. This would not be expected by the model 
and it poses some- inicrosting possibilities. Perhaps, teachers who fre- 
quently test spcMid less class time tutoring or teaching or adapting assign- 
ments to meet Midivulual needs. Or perhaps frequent testing of a child 
in and of itself is dysfunctional to increases in learning. In either case, 
it is llu* type of information which is important to verify and feed back to 
developers iind implc mentors. 

In addition lo its analytic function, the instrument was to provide 
information about the implementation of the program in the field and to 
ptMinil a iDinparison of the implemt»ntation between the field and the 
iaboralory siti»s. It would be impossible withir^ tho scope of this paper to 
pres(Mit a detailed tlescription of each classroom on each variable. Instead, 
two descriptions will be provided; first, a comparison between the field 
and labiMMlory sites and. second, a general discussion of how the model 
looKs in all ilu' t la s.sroiuns examined. 
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\ TABLE 5 

Partial Correlations Controlling for Entering Abilities 




Class enrollment 


1.00 


-.27 


-.34 


.17 


.11 


%30 


Ratio of boys to 
girts 


.27 


1.00 


•72 


.22 


.36 


•30 


Time between tests 


.34 


.22 


-1.00 


.00 


.00 


\ 

.32 


Percentage of 
negative contacts 


.17 


.22 


00 


1.00 


.19 


-.37 


Amount of time 
spent on mathematics 


.11 


-.36 


00 


-.19 


1.00 


-47 


WRAT math means 


-.30 


.30 


.32 


.37 


.47 


1.00 
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Table 6 shows the site averages'^o/i seventeen variables, 
paring (he laboratory and field sites, those variables which appe 
lar in both field sites and developmental sites* will be discussed fir 
followed by those variables for which there is a difference between t 
developmental and field sites. 

The variables on which there is similarity are teacher experience^ 

with the Li^DC Model, sex ratio In the class, percentage oif unique 

assignments, percentage of negative contacts, access to play following ^ 

« 

work, the number of adults traveling, and the number of minutes of math 
or reading per day. For most of the variables, the field and develop- 
mental sites look similar, s 

The variables on which there is a difference, however, arc quite 
interesting* The njimbcr of pupils per class, especially when considered 
in light of the perientagc of children present is smaller in the develop- 
mental schools 111. in in the field site schools. This is an important difference 
when examining outcome measurers and per pupil expenditures. There 
arc other differences; in general, more time between tests elapses'for 
those in developmental schools, they make fewer cognitive contacts and 
more management contacts, and the teiicher attention was distributed 
more evenly. The converse of all these findings is true fo.r the field sites. 
U would appear that m some respects, the field schools perform in a way 
that more closely resembles the expressed model than do ^the developmental 
schools, A possible explanation of this is'that developers are constantly 
changing Ihc model in the s.chools to which they have access, placing 
demands on the teachers so that the classrooms do not strictfy follow 
written or spoken guitle lines, 

. A more relevant question than the comparison of field and develop 
mental si hodls is how much do the schools look like the model? The 
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answer cannot be given a precise value, but some^gcrreral obBo'rvatlohs 
can bo made. If one pbserves the Follow Through classrooms using the 
^ LRDC program to determine if they operate in a manner similar to, 
traditional class'roorhs, the answer is clearly no. All of the classrooms 
visited arc individualized to some extent. Some sites have modified the 
model th a specific wijty, such as having two teachers travel, bthcrs have 
extended the iSlanagcmcnt system from the LRDC program curriculum 
area to all curriculum areas. Some classrooms lest frequently, but 
tend to jjivc chihiren simUar assignments rather than individual ones. 
No classroom assigns juyt rhose pages needed by the child as indicated 
by ihe test; rather, most of Ihcm start at the first page a child needs as 
indicated by a feebt and assigns a block of pages after that. There are at 
least four different styles of traveling, all of which av^a compatible with 
ih^^ l.RDC Instructional Model. 

Whnt is t|ear from Table 6 is that the program can be implo- 
merited In diverge settings. It is also cleat^thJit the program undergoes 
a certain amount of n1odifi(^.llion in the field. One of the que stjions- raised 
by a Bttidy like this i^ what«modifi cations in the model, made in the field, 
iinprove the model. Far exaniplc, perhaps the model should place less 
cMnphasis on frequent testing and unique prescription and more emphasis 

on differing n)c>tles u( transmitting information^ Hn a well sequenced cur- 

■I 

ru ulum It may not Ije nect^ssary ^to continuously monitor progress;* and 
^ tin? same effort may be b'Uter expended to provide a diversity of curricu^ 
fum objijctives and a means to moot thom. 

♦ • 

Limitations 

The mstrumopt doe's ndt prcjvicle information on all of the domains 
initially identffiod. Some variables were very difficult to measure with^ 
out extonsjvo ( I mica! data or without developing separate measurement 
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!n3tfurmnl$ for them (c,g, , student independence). Sonic variabj. 
were Ipsl because of the inability to obtain the info^rmation in a reliable 
fashion/ Still other variables were omitted *be cause there appeared to 
be no measured difference between classrooms. However, a good st^rt 
has* been made in the construction of a reliable and valid instrument for 
measuring classroom process and the impl<?meritation of the i^lUDC 
Ihsti*ucti5nal Model. 



implications r / 

An implementation instrument provides information al)out the 

edu^cational processes that occur in class i*ooms using a particular inno- 

vatioti. The jnstrunnent can be a valuable tool to o'valuato^rs -examining 

the overall results of an innovation in explaining those 'rcsulls, but itOs 

also a useful tool for implementors and developers. For implcmohtor s, 

it provides information about the success of the implementation relative 

to the model. Foi* developers, the instrument can provide information on 

the conseqifl^nces. both positive an^ negative, of unintended changes in 

the Inodel. This infornmtion in turn can become the basis for change in 
♦ 

specific programs and overall assumptions of an educational model. 
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THE USE OF OBSERVATIONAL DATA FOR 
. FORMATIVE EVALUATION OF AN INSTRUCTIONAL MODEL^ 

Margaret C. Wang- ^ 

Learning Research and Development Center 
University of Pittsburgh 

* 

This paper discusses the critical functions classroom observational 
techniques can play in the formative evaluation of an instructional model. 
The study reported in the paper is part of an ongoing series of studies 
designed to develop techniques and procedures to obtain information for 
formative evaluation of an early learning environment, designed to adapt 
to the individual learning characteristics of the student. Specifically, 
the study was designed to gather descriptive data in classrooms settings, 
under two different instructional-learning management systems designed 
to implement the LRPC Instructional Model (see Appendix A) in the early 
li^arning grades. Our aim was to identify techniques and measures that 
are effect ivc in obtaining information about the differential effects of 
instruelional-lt^arning management systems on the instructional-learning 
processes and studonl learning outcomes, 

ihe techniques and measures developed for the study included the 
use of systematic observational schedules designed to gather information 

-* 

^The research reportecT'iierein was supported in part by a grant 
from the Ford Foundation, and in part by a grant from the United States 
Office of Education to the Learning I^esearch and Developmiint Center. 
Tl]e opinions expressed do not necessarily reflect the positiof\ or policy 
of the sponsoring agencies and no official endorsement should be inferred. 
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concerning classroom instructional-learning processes of the student, as 
well as an observational technique designtid to obtain detailed narrative 
descriptions of the learning environment and classroom behaviors. 

The -Setting 

The study was carried out in a first grade classroom of an inner- 
city public eleitientary scho il, serving as one of the LKDC developmental 
schools. The first grade, operating under the LRDC InstruclMonal Model, 
included two major components, l^e "prescriptive" learning component 
and the "extended" learning component. The prescriptive learning, cqm- 
ponent of the program included math, reading, and perceptual skills. In 
this component the students were assign(?d to learning activities on the 
basis of formal diagnostic lest results. Kxfcyided learning activities 
encouraged student initiated activities. These generally included more 
open-ended projects in such sul>ject matter areas as math, science, 
social s^tudies. n?adin«4, writing, prereadinu, and language arts; in addi- 
tion, they included a variety of creative arts activities, construction 
activities, and conceptual games. 

7\vo prototype! ins! ruct iona) - lea rnmg nianagi'^nienl syst(.'ifis were 
desii*>1ed to implement the LRDC Ins'. r\K Monal Model in school settings. 
The nuinai4iMnertt systems were? the l>lock Schedule Sysit»ni anfl the S<»lf- 
Schedule System. Hofli systems were designed to implenu'Ut n\\ aspects 
of the l.UnC Inslructi'Mial Model for early learning grades (Wang, Ma/.^a, 
Leinhardl. 1^ Millmorr. 1971). (?nd"rthe Mlotk Schedule System, the 
school day is divided into lime blocks. In ueneral, the class begins with 
bloi k periods for "prescriptive" learning, followed by a tjlock period for 
* 'ey ' entlefl" learning* For first grade. I he prescriptive learning block 
pt ds are readinj^* iDath. and perceptual skills. The extended learning 
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period is usually scheduled at the end of the school day for th c wl»o 
complete all the prescriptive assignments for the day. 

Under the Self-Schedule System, as in the Block Scher?' Sy. m. 
the student, bcirjed on the diagnostic test results, v^eceivcs c}..tly ct tign- 
ments (prescriptions) in reading, math, and perceptual skiJJ However, 
no specific, time bloc^ is assigned for a specific type of learnin*^ 'ivity. 
The students can work on the learning activities prescribed by the teacliCi 
or on any extended lear^ning tasks of their own choice at any t inu . Under 
the Splf-Schedule System, the students. arjS given the opportip ny to nuK« ^ 
their own decisions on \yhen they will do what, with the c:.< ejitioi. mat 
some parts of the wh at are prescribed by the teacher, 

K4e t ho d 

De sig n^ A repeated-measures design was used for this atuJ •? The 
Block Schedule? System was used during the first four mouths of 1 .school 
year (considered as the baseline peri<^[B|)), although aciualYlai i eojlcc- 
tion for this perioci did not begin until the third month. The Seif-Srhedule 
System was used during ihe four months immediately following Bj (con- 
sidered as the experimental [E]), The Block Schedule System was used 
• again during t!i(^ reversal period (B2). which lasted one monlli i. mediately 
following the experimental period. 

Sampl e, All the students of the first grade class selecli d for this 
pilot study served as subj^^cts. The total number of students n;»:n^-d in 
Ihu class wns The mean chronological age o£ the students during the 

experimenla) prriod was six years and seven months, Tlte meap IQ for 
llie j^roup was *^7. and the standard deviation was 11.7. TI10 students 
came from an inner -city neighborhood, with the majority of them from 
low income faiijilifs* 
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Data^athermgjecji^^ A systematic observa- 

tional schedule was designed to gather information about student learning 
processes and patterns of interactions in classroom settihgs operating 
under the LRDC Instructional Model. Hie Student Behavior Observation 
Schedule (SBOS) was designed to obtain answers tp the following questions 
(see Appendix D for an example of the SBOS): 

1, What type of activity is the student engaging in (prescriptive 
or exploratory)? . ' 

2, Under what conditions does he choose to work: 

a. Does he work with others interactively? 

b. Does be work on tasks in isolation'' 

3, Does he complete the activity he chose to do first or does he 
switch from one activity to another without completing the ini- 
tial one ' ' 

4, How did he decide on what he wants to do-"assigned by the 
teacher, by invitatiorTfrom another student, or self invitation? 

5, What IS the major tone or the manner in which the task was 
carried out- -aimless , purposeful, or inattentive "i* 

6, What is the predominant w^ode and purpose of his interactions 
with the teacher*' 

7, What is the predominant mode and purpose of his interactions 
with other students in his class*^ ^ 

The SBOS wks designed to code ongoing student classroom behav- 
iors. The schedule consists of one recording sheet, which includes 
seven major behavior categories, and spaces for recording five time 
samples of behaviors of a single student. An observer instructional 
manual, which includes definitions of the major behavior categories and 
the behavioral subitems listed under each category, was also developed 
for training purposes (Wang, 1972), 
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The technique of narrative recording of student behaviors was also 
used to gather information for the study. The narrative recordings were 
used: (1) to study I in depth, the differential effects the different manage- 
ment model has on individual students uf certain known unique character-, 
istics: (2) lo find out whether Ihe systematic observational schedule alone 
is adequate to provide the desired information; (3) to compare the obser- 
vational data obtained from the SBOS with the data obtained from the more 
comprehensive and more detailed narrative recordings of behavior; and 
(4) to determine whether the narrative recordings indeed contribute addi- 
tional pertinent information about student learning that cannot be obtained 
from Ihe SBOS. ^ 

The narrative recording of student behavior focused on student 
beTtavior i*elevanl lo two major questions: 

I. What are some of the noticeable differences in the student's 
behavior that can be attributed to tht? rftanagement system? 

a. Choice of activity, 

b. How chosen'* 

c. Whom he/ she chose to work*\vith, 

d. Manner^ in which activity is carried out, 

e. Predominant mode of interaction, ' 

Z. What arc some of the learning outcome? produced by this par- 
ticular management system ' 

a. .Student's ability to make choices, 

b. Student's ability in plan his day, 

c. Student's ability to meet his curricula r demands. 

d. Social interaction with peers. 

e. Particular achievement in prescriptive and extended 
learning, ^ 

f. Attending behavior. 
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tn addition to the observational data, we also kept a record of the 
total number of prescriptive learning and extended learning tasks copi- 
pleted during each school day for each observational cycle of the study*' 

P roced ure. Each student in the class was observed for IS minutes 
at a time for a total of 75 minutes during Bp E, and B^. A specific 
observational schedule was established to insure that a l5-minute obser- 
vation out of each different hourly segment of the school day would be 
I lade for f»ach student. The school day was divided into one-hour seg- 
ments* The 75 minutes of observation for each student occurred over 
two to three days and in no case was a given student observed (wice 
wilhin two consecutive hourly segments. Under this stipulation, four 
different students were observed during each hour. The students were 
r:» iked alphabetically to establish the order of the observation sequence. 
The observer A)llowed the established observation sequence as closely 
as {)ossible. Occasionally, because of absences and other unexpected 
^events, some minor modification in the observational sequence was made. 

T\}c observer focused on one student at a time. stopwatch was 
used to time t\ie observation intervals, and a frequency coding method 
was used to record behavior occurrence. The 15-miM'ito observation of 
each student was broken down into a minute of observation, during which 
the ob :erver coded whatever behaviors she observod occurring under the 
two interactional categories as listed in the SBOS (catt ><ories and 2^), 
foHowt d by a two-minute rest period, during which the observer made 
appr( priate entries for behaviors listed under other categories (2 through 
7). .vctual observation time during each 15 -minute observation period 
was on minutes 1, 4, 7, 10, and 13, a total of five three-minute obser- 
vation intervals, the end of the 15-minute observation of Student I, 
the observer began a 15-minute observation of Student 2, following the 
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same procedures. The observer used a new recording sheet for each ^ 
new 15-mlnutG observation. ^ 

In addition to the observations made for every student included in 
the sampli*, a group of six subjects from the sample were selected, for ^ 
intensive study. These six children were selected on the basis of thfeir 
learning progress in prescriptive curricula and teacher judgment. On^ 
girl and one boy were selected from each of the student achievement cate- 
gorles- -advanced, average, and below average achievement. Narrative 
description of student behaviors, as well as the SBOS, were used to obtain 
inforjmatlon about the six students in the intensive sample. 

During each of the three observational cycles (Dp E, and Eg), in 
addition to the data obtained from the SBOS for each student included in 
the intensive sample, narrative description records of a 15-minute obser- 
vation period per day were made fbr five consecutive days. Therefore, 
counting both types of observations, we have collected a total of two and 
one-half hours of observational data for each student included in the inten- 
sive sample. To insure that the time samples for any given student were 
taken from different times of the school day, a schedule was designed to 
systematically alter the time of the day during which the additional 15- 
minute observation for each student was made.. 

During each 15-minute observation tirr.i! segment, one child was 
observed at a timiv. the observer first used the'SBOS to code behaviors 
as they occurred during the first minute of each thre^-niinute interval. 
(The procedure used during the first minute of observation is the same 
as^e procedure outlined for obtaining data from the SBOS for the total 
sample. ) During the second and third minutes of each time interval, the 
observer, using the preplanned guidelines for the narrative description 
of the student behaviors and learning conditions, wrote down all the 



pertinent facts for the first minute, using an improvised shorthand method 
developed by the observer* In the same fashion, the second observation / 
interval began on minute 4. The same procedure Is repeated for the 
remaining intervals on minutes 7, 10, and 13. After each 15-minute 
obsei*vafcion of a given student, the observer used as much time as she 
needctd to add to the narrative description ai ' behaviors and facts 'that 
occurred during each of the fdve observation intervals. The same proce- 
dure was followed for each 15-minute observation for the entire intensive 
study sample. 

D ata Analyses 

Observational data obtained from the SBOS were analyzed under 
two main classifications: the student learning process and the student 
interaction patterns. Observational results of student learning processes 
arc summarized in Table L. r» 

The inter-observer agreement was highly consistent for all but Qne 
behavior item, "group interpretive" under the behavior category entitled 
"setting. " (The reliability coefficient was . 60. ) The low reliability co- 
efficient in this case probably resulted from a combination of the infre- 
quency of occurrence of the behavior, and the difficulty in determining 
the nature of behavior in the group setting?' It was difficult for the obser- 
vcr to get N^ithin the close physical range required to determine accurately 
whether the subject was actually interacting with other students, or just 
working in a parallel fashion among othnf students in the same work area. 

For each of the observational cycles. Baseline. I (Bj), Experimen- 
tal (E). and Baseline 2 (B^), the mean ratio of behaviors that occurred 
for bach of the behavior subitems was calculated* The results are sum- 
marized in coi imns 2, 3, and 4 of Table 1. The ratio is obtained from 
the total number of behaviors observed for a given behavior subitcm* 
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divided by the total number of minutes of observation time. To tedt the 
significance of the difference between the behavior measures obtained 
during each of the observational cycles, a t-test was peifformed between 

o t 

the moans of each pair of observational cycles* The results are reported 
in the last three columns of Table !♦ 

As indicated by the reaults, the different instructional-lea^rning * 
management systems seem to have differential effects on student class- 
room activities. '^The data showed that under the Self-Schedule System 
(E). students worked significantly more on extended tasks of their own 
choice, and worked more in group settings; the frequency of prescriptive 
activity decreased significantly, most of their activities being self- 
Selected and self -initiated rather than assigned: and the students were 
more purposeful and Spent significantly less time waiting for interven- 
tion 'from the teacher. However, no significant differences were.found 
in the observed number of activities completed, or in the number of tasks 
left unfinished. The "no difference" result of these two behavior cate-t ^ , 
gories n. ly be attributed to the* "unreprasentativeness" of a sampling 
technique for bbserving behaviors that genei'ally occur inf r equently<# 

The data on student behaviors also suggested that students* in 
gener44i}*bchavcd more similarly during the two baseline periods (using 
the Block Schedule System) than they did di^ring the experimental period 
when the Self -Schedule "jTy stem was implemented. For example* students 
worked significantly more frequently on prescriptive tasks* they were 
more inattentive and less purposeful when they worked, and they exhibited 
8i^nif^cant ly more "waiting" l^ehavior during both baseline periods than 
they did during the experimental period. 

Patterns of student classroom interactions under the two different 
insftnictional - learning management systems are summarized in Table 2« 
The results are displayed in the same way as Table !♦ About two-thirds 
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of the inter -observer rcliabiUty cooffici'-ni ! ?r sIhcK-mi cl . ».-i«) jni iuler* 
acUons vvere within a range clorfo to . I'lu .iboui oi^-^ Or.:- n' lht» beh'.u - 
lor items showed low consislcnt y bctwct . Mljgorvi'rs. .^.t hAvicral itcmji 
with low inter-obscrve r rcliatjility coefl.^K-Mts v^i vr nonverbal inv* ractiona 
wilh peers, and the behavior subitems undor the general heaclinj^ of pur- 
poses for peer interactions. We liave reasun tu bi-lieve that the luw relia- 
bility coefficient of these behavior subiti-ins l.irgely attributed tu the 
difficulties in setting both observers simultaneously within a close vangc 
of the siibject being observed. This is? a major technical problem one 
i(enerttlly encounters when conducting observational sindi^^s in natural 
settings, and this problem is particularly magnified in classroom set- 
linjis where students are permitted to freely move about the classroom 
and interact with their peers. According to the feedback information 
from the observers, the magnitude of this problem decreat^ed when class- 
room observations for the study w'ere made; it was much easier for one 
person to get .to a vantage position to^focus on the subject being observed 
than to get two persons to approximately tKe same position in order to 
observe the same student at the same time for th^ reliability study. 

The data on student interaction p^^tterns were consistent with the 
other findings on student learning processes. Tlic'students behaved dif- 
ferently under the two different management systems. By examining the 
mean frequency of brJiavior occurrence under the heading of student- 
teacher interaction one notices two things: (1) that students interacted ' 
with the teacher more often under the Block Schedule System than during 
the Self-Schedule System, and (2) that teachers initiated significantly 
more of the interactions during the Block Schedule System. However, 
Mu) significant differences were found in the purpose of the interactions* 
rije observational data on student interactions with peers showed that 
students interacted with one anSther more often under the Block Schedule 
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f lern thtin in 'h*- 



I -Si.SrcU>lt* Syj-lcirj. However, it i.4 Inienrsling to 



* ..e that a larj..- ut" * : ; i%of r interactions under tho Block Schod - 
ulc System uerr dt -a rr'-irunii" and 'conversational^* > 

!n j»enerti!. iht rt^Milts -mv, that the .SJ30S seems to U- an effec- 
tivc instrMrnen} tt> u»s(. vn rlocur>»oni ihe behavior differences in student 
learning processes hikI interaction patterns in classroom settings. Fur- 
thermore, the buhavlot>calt-i;ories included in the S DOS were relevant to 
the rationale and hypolhesew made on the tea^hin^ -learning processes 
under the two different instructional-learning management systenis. The 
data yielded information that not only made analysis of ^he. selected dimen- 
si'ons of student classroom behaviors unique to our program possible but, 
more importantly, the data provided us with information to carfy out 
formative evaluation needed to improve and redesign the instructional- 
, learning management systems. an example of the use of empirical 
data (data obtained from a systematic iitudy such as the present study) for 
formative, evaluation purposes, I shall discuss the result of our analyses 
on the effectiveness of the two different management systems in solving 
the proolem of student "waiting time, " ^ 

Among the many technical problems we encountered in implementing 
the LRDC Instructional Program was the demand on teacher time. The 
teachers were not atlo to get to the students fast enough to both answer 
questions arffl to check their v/ork at the same time. Under the individu- 
alized instructional program* students generally spent a large percentage 
of class time "waiting'' for teachers instead of working on the tasks. The 
observational data from the present study, as well as results from previ- 
ous studies, clearly suggest this unique "waiting" phenomenon (Wang, 
Mazza, Haines, L Johnson, 1972; and Yeager k Lindvall, 1968). This 



"waiting time" was generally a frustrating experience for teachers and 
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s^tudents. For eludents, the "waiting time" was tiresome and boring. 
Studenls were not only bored from doing "nothing" while they were 
waiting, but a^so from doing the same type of tasks for a long periorf 
of time (e.g. , some students may be required to work on prescriptive 
tasks from one subject ^o another during most of the school day). For 
the leaoher, the jstuderjt "waiting time" was "trouJjlesom<^" and "hard to 
manage." The teachers were required to' "travel" among the students - 
at a fast. rate in orde;r to get to every student who needed>help, and also 
to keep those "waiting" students from disrupting others at work. 

' The results of this study suggest that the Self-Schedule System is 
effective'in reducing student "waiting time" to a minimum level. As 
shown in Table 1 (p. 59), the mean ratio oi "waiting" behaviors decreased 
from .25 for Bi to .08 for E (the t-test was significant at\thL* .05 level). 
Although the difference?; of Ihe percent of waiting time between E and B2 
were not statistically significant, the differences ivere still substantial. ^ 
Waiting time for was . 15, almost double the amount of waiting time 
during E.' Frorrva formative evaluation point of view, this preliminary 
set of dala suggests that the Self-Schedule System may be more effective 
than the Block Schedule System in solving the problem of student "waiting 
time. " ' ' 

Our data for formative evaluation of the instructional management 
systems were also used to study the relationship between- learning proc- 
esses and learning outcomes. Canonical correlations Nvere performed 
to investigate the relationship between student -classroom behaviors and 
student Iea?rning outcomes undef* the two different management systems, 
tn this series 6f analyses, we were particularly Interested in*examining 
the relationship between a linear comb^ination of tasks completed (tasks / 
Included n both prescrip ive and extended learning), and a linear combi- 
nation of selected sets of classroom behavior measures. The results of 
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the canonical correlations are shown in Table 3. For each aiialysis, the' 
variables included in the criterion set are listed under the first column; 
the variables included in the predictor set are listed under column 2. 
The canonical 'R, chi square value, degrees of freedom and the statisti- 
cal test of significance are reported in columns 3, 4, 5> and 6, respec- 
tively. 

The criterion set included in Analyses A and B was mean number 
of tasks completed in prescriptive and extended learning during Baseline 
1 and the Experimental periods. The predictors included in the analyses" 
wiire seven selected behavior measures obtained by the SBOS. These 
measures included variables that we have suggested as critical learning 
process variables that affect student learning outcomes under the LRDC 
Instructional Program: (a) prescriptive activity, (b) extended learning, 
activity, (c) self - initiated activity, (d) purposeful on-task behavior, 
(e) waiting for teacher intervention, (f) sharing behavior, and (g). con- 
versation with other children. 

t 

The rationale for performing thls'^eritis of canonical correlations 
is based on our assumption that what the student did and the manner in 
which he carried out the learning tasks must have direct implications 
for student learning outcomes* As shown in Table 3, a significant rela- 
tionship between the task completion rate and observation measures of 
student learning processes during the Self-Schedule System was found 
(during the experimental period), while the relationship between what 
^\X^ students were able to do and the process under which the tasks were 
ciirried out during the Block Schedule System (daring the baseline period) 
was not significant. 

The results suggc^st that behaviors exhibited by the students during 
fho rxperimenlal period seem to relate to students' task completion rate, 
while students' behaviors under the Block Schedule System were not 
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related to what they did. This interpretation is further supported by the 
fact that students spent significantly more time under the Block Schedule 
System doing prescriptive worK» without completing a concomitant amount 
of prescriptive tasks, and the fact that students under the Block Schedule 
System also spent more time "waiting" for teachers. In other words, 
the canonical results for Analysis A, listed in Table 3, seem Ito suggest 
that student behaviors under the Block Schedule System were not related 
to their task completion rate. The fact that the process measures ob*tained 
for the SBOS have been good predictors of student learning outcomes under 
the Self-Schedule System, but not related to learning outcomes under the 
Block Schedule System, further supported the assumptions we have made 
in the design of the Self-Schedule System, However, because this is a 
pilot study with a small sample, these interpretations should be treated 
as tentative. 

Another example of formative evaluation questions we asked from 
the data in the pre Study was, "Is the SBOS an effective instrument 
to use for evaluating the learning processes of individual students? " To 
answer this question, we examined the observational data obtained for 
each of the six students included in the intensive sample individually. 
In addition to the observational data, to investigate the learning processes 
of each of the six students, we also collected information on a selected 
number of student learning characteristics. The student characteristic 
measures are summarized in Table 4, The behavior changes of each 
student observed during each of the observation cycles, using the SBOS, 
are summarl/,ed in Table 5, Behavior items included in Table 5 are 
behaviors found to be significantly affected by the different management 
models (significantly different p^irs of means as reported in Tables I 
and I [pps. 59 and 61. respectively]). On the whole, the data suggested 
that iht' different management models had differential effects on student 
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behaviors* We were able to obtain a general picture of the diff< rence in 
bohavlcrs and uulconitjii for each student. However, differential eifects 
of the different syBtems on individual students were not as distinct as we * 
had hypothesized, Tlie general group trend of behavior changes under the 
different systems prevailed. 

The Self -Schedule System was consistently effective in promoting 
the student's rate of prescriptive task completion (see Figure 1), but it 
did not affect the rate of extended learning task completion (see Figure 2), 
As shown in Figure 1, every student donslstently had a lower rate of 
prescripjtive learning and task complet.ion during Bj and and a sig- 
nificantly higher rate during E, Some differential effects on the rate of 
extended learning task completion of the individual students can be 
detected in Figure 2, However, because of the small number of extended 
learning tas^s completed by each student during the different obsei^vation 
cycles, the magnitude of the differences was quite small-«the differences 
were not statistically significant. Nevertheless, when one examines 
Figure 2 closely, one can see the differences in the trend of behavior 
changes. For example, the effects of the system on Students and E are 
different from that of Students B,^ C, D, and F. ^Comparing Figure 2 
with Figure 1, the different management systems did not seem to have a 
consistent effect on sfudents' extended learning task completion rate as 
they had on students' prescriptive learning task completion rate. 

As we analyzed the data oL j . from the SBOS for ^e six students 
included in the intensive study sample, the lack of "depth" in the data 
became evident. This supported our notion thaf observational data 
obtained from a measure such as the SBOS do not yield sufficient infor- 
mation about the individual student in order to carry out formative evalu- 
ation of individual learning processes and outcomes. * 
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Figure 1: Presciiptive leurtiing Tasks Complotod. 
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To answer the question of whether narrative description data pro- 
vided additional information for studying, In depth, the differential effects 
of the management model on the individual student, the combination of 
observational data obtained from the SBOS and the narrative description 
data of the six students included in the Intensive study sample were ana- 
lyzed. The narrative records of each student were first compiled and a 
summary of each student's description wais made. The behavior descrip- 
tion was then compared with data obtained from the SBOS tor each stu- 
dent. 

The Information obtained from the two^different sets of data were ^> 
found to be in general agreement; there were no sf^eciflc inconsistencies. 
However, the data clearly reflecteo .»at the information yielded by the 
two observational techniques served two distinct purposes. The SBOS 
is effcijtive in foVmative evaluation of the instructional model, but not 
sufficient for formative evaluation of the learning processes for individual 
students. The use of the narrative description method provided more in- 
depth information about the individual differences In learning processes 
among students. 

Discusstbn 

Through the use of the Student Behavior Observation Schedule (SBOS), 
we were able to obtain pertinent information about the relationship between 
the LRDC Instructional Program and its specific effects on student learn- 
ing processes and student learning outcomes in classroom settings. We 
found that obseVvational data of this kind is useful in delineating the ,rela- 
tionshlp bet>frecn wlftt the instructional model requires the teacher to do 
in the implementation of the LRDC Instructional Program and the effects 
on the students. We found, for example, that the Self-Schedule System 
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produced: (I) an increase in the rate of pre^rcriptive task completion per 
weeljl^ although students worked on pres<?rlptive tasks less frequently; 
{2J more purposeful and attentive behavior on the part of students; (3) a 
decrease in the frequency of student Waiting behavior; (4) an increase in 
the frequency of students working in group settings; and (5) decrease 
in student "disagreement" behavior. 

More importantly, the data from the SBOS were useful in giving us 
the feedback information about the aspects of the instructional-learning 
mana^^ement program that did not function as hypothesized* Information 
of this typ6 is critical to the validation and revision of the design of a 
management system. The data indicated, for example, that the Self- 
Scltedule System did not effectively increase the rate of task completion 
in extended learning, even when it was observed that students engaged 
in extended learning activities more frequently. Further investigation 
of the results suggests that using the rtite of task completion to evaluate 
extended learning outcomes is misleading. The fact that student ft id Jiot 
complete more extended learning tasks (as measured liy number of tasks 
completed) does not ^necessarily mean that students were less efficient in 
task completion under the Self-Schecfule System. The results may mean 
that students spent more time working on a given exiended learning proj- 
cct, thus completing fe ver total tasks. Furthermore, because of the 
varied nature in the quality of student invo/vement in the extended learn- 
ing activities students have choscn.^to do, comparing outcomes in terms 
of numbers is neither sufficient nor appropriate. Perhaps using both 
"length of time per activity" as well as "number of tasks completed" can 
provide us with a more adequate measure. It is probably more meaning- 
ful for a student to spend a i^elatively longer time to accomplish an 
extended exploratory project of his choice, than completing more ^hort- 
term learning projects. Nevertheless, these assumptions will need to 
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be studied before we can make a defi^U'fe conclusive statement. The 
results, however, demonstrated the need J^o use a combination of many 
different types of inXormation to conduct formative evaluation of student 
learnigg* 

Wc also found that the Self -Schedule System did not increase the 
frequency of peer intoracUon as we had hypothesized. 'ITiis finding 
implies tUat a hiore thorough examination of the type of -tasks We have 
available in the learning environment is necoss<\fy. The fact that these 
suidents worked significantly more frequently in group parallel settings 
vftder the Self -Schedule System suggests that the management system 
probably was effective in providing the opportunity for the students to 
interact with one another; but the nature of the tasks available to thetin 
may not reinforce and/or permit interaction among the students. Tliie 
result also illustf^Ttes the multiple functions that observational data of 
this type can servi>. The observational data provided us with the forma- 
tive evaluation information of our instructionaUlearning content, as we^ll 
as information about the instructional- lea rning management system. 

Another important input from thc^ observational data concerned the 
teacher-student interactions. While the frequency of interaction bj^tv^n 
the teacher and students did not increase, student waiting time decreased. 
The results can be interpreted aB an indicator of the effect i'vent^ss of 
the SeU-Scl^(Jule System to achieve our objectives^-to cut student waiting 
lime to a minumum and to enable the teacher to have a maximum amount . 
of interaction. Although the data do not ahow an increase in the frequency 
of teacher- stvideyl. interaction as We have hypothesized, we Have reason 
believe that the am,,>unt of time per interaction involved has increased 
under the Self -Schedule Sv stem. Another implication that our data sug- 
gest is that an increase in the ratt> of task conVpletion is the result of an 
increase in the .amount of time per interaction, rather than in the frequency 
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of intcractione, A system designed to promote frequency (the rate)..of 
teacher -student interaction, theoretically, should look very different 
from a system designed to promote extended interaction between the 
teacher and student. 

Summary 

This pilot study has shown that classroom observational data can 
oonlrlbute to ihe formative evaluation of the instructional model, as well 
as to the formative evaluation of the learning processes and outcomes of 
individual students. The data obtained for the six students included in 
the intensive Sijimples provided Ub with pertinent information about dif- 
ferentia! effects that the management system'can .lave on individual 
studentiu To achieve our goal of providing a truly adaptive learning 
environment that takes into consideration all aspects of individual stu- 
dent learning needs, information about the eflects of certain types'of 

ujatructional-learning management systems is a firerequisite. 

* ■ . " 

The following summary of interview comments by tlie teachers 
who were awa^e of the type of information we obtai Jtcl from the observa- 
tional study illustrates thi > point: 

The teachers spent quite a bit of time discussing the different 
ways that kids respond to their learning tasks. They found 
that some students consistently chose to work on ^<xtended 
learning tasks first, and prescriptive learning tasks later. 
Others do prescriptive learning tasks first and extended 
learni-ttK* tasks later. Still others intersperse the two. 
Tliey have noticed that students who have the most energy 
and are most likely*to be "messing around" are the ones 
Mvho spend the most .time n extended learning activities* 
This "energy" dimension seems lo cut across academic 
ability or a^l least academic standing. They discussed 
specific students and how they fit into these general pat- 
\, tfti,i8, T^ey stated " , • , students who are really task 
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oriented and are not too fast, like D^and E, extended learn- ^ 
ing u^ed ^o end at approximately 2:10 or 2:20 and they wou)<^ ^ 
barely get into extended learning activities before the day 
was over. . . . But they were task (prescriptive task) oVi- 
entd^l, and they would just do thxit first . . . R has been doing 
her two.malh boxes very quickly in the morning and I will say 
something like 'Oh. good; now you can get your reading done, « 
and she will say, »No, I'm going to extended learning activi- 
ties now, and then*Pm doing-my reading because that's, the 
way I'm doing, things now. ' ^ . The students who would- • 
spend a lot of time in extended learning cover a range; B 
and T would happily leap around all day and so would S and 
* D who are at the other end as far as ability and what kin ' of 
work they have done. . . - 'Kike T ... we can see he will 
get his work done. He will work on iwo boxes at a'time. 
lie is -not going to wait for us to come around. . • That 
is wasting too much time; he cannot get done fast enough. 
. , The teachers also talked about the 'importance of . 
interactions among students as an important measure of ' 
. Mic^uccess of an activity, in addition to the appropriate- 
ness an activity for a particular student. They expressed 
^ their pleasure at seeing M and T working together on build- 
ing projects. They felt that M was iiaving a unique learning 
experience building with the much more veKbal T. Tliey 
valued P's * Uerest in listening to the record player, R 
teaching others how to knit, students playing a card game 
together, complex block constructions from T, clay crea- 
tions from M, and sen«;ory play with seeds for T. 

It is exciting to know that teachers can have information regarding 
the student's distinct stylistic differences in the ways they carry out 
learning tasks. ?>uch information is a useful contribution to work on 
adapting instruct;onaMearning processes to individual differences. 
The Information forces the teacher to pay attenHon to the processes 
used by the individual student, rather than locto.ig only at outcomes. 

The results of the study clearly suggested the potential usefulness 

e 

of the obftcrvational techniques in obtaining pfcrtincnt evaluation informa- 
tion. The observational data provided us with the information 'we needed 
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6 investigate the Velalionship botween the* instructional-learning systcm(s) 
sed r .i.iplement our instructional program in classroom settings, and 
what Sjppens to the student learning processes and student learning out- 
comes. However, as I have pointed out in the discussion of results, a 
great deal of validation work still needs to be done before wide applica- 
tion of the techniques and measures can be used in classrooms using the 
LRDC instriicUonal Program and in other classrooms using different 
inAtruct ional approaches, , 

As we develop relrablu and valid techniques and measures to exam- 
ine the classroom learning processes and outcomes, and as we are able 
to obtain more formative evA*luation information I have described, we 
will be )3etter equipped to design a learning program and learning environ- 
ment that will be adaptive to thu multiplicity of factors and student charac- 

0 

teri»^icfl that affect student learning in school. It should be our objective, 
as pro)kra'm developers^ to provide teachers with techniques to help them 
to diagnose individual differences and needs of their students in dimen- 
sions oth«Vjhan academic achievement. Developers should also equip 
teachers with wayjr to make some of the program objectives more con- 
crete.^ Wo will be able to make the difficult job of teaching in an indi- 
viduaU/.ed insl r\Kl ional program less difficult and more explicit, to the 
teacher only when we can clearly define the instructional-learning manage- 
ment system, Ihe learning conditions, and what we can expect to obsorve 
in the nature of the interrelationship between teacher behavior, the -instruc- 
..tional-learninK management system, and student behavior. 
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I.KAHNINC ACTIVITIES IN 
INDIVIDUALLY PRESCRIBED INSTRUCTION' 

o 

Joseph Shimron 

Learning Ro"search and Development Ct-nter 
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A t.-ajor goal of the Primary F.<lucation Project fpEP) and Lndi- 
Aidually Pr.Vs'cribod Instruction (tPl) programs (see Appendix A) is to 
adapt the kiucational syBtcm to individual differences. The extent to , 
which thi8 adaptation occurs is under continuous study and evaluation. 
The major p..r,.oses of thiv. pilot study were: (1 ) to determine whether 
or not dislmct'ive patterns of behavior exist for students of different 
learning characteristics (e.g.. learning rate), and (?.) whether or not 
and the degree to wfrl. h tht*" IPI system succeeds in adapting; its instruc- 
lion.il procedures to these behavior patterns. 



Method 



M^sjiXc^s^of AdaEL^biliiif. The question posed is how well the IPI 
ystem actually adapts to individual differences. In order to measure 
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adoptability, one can observe rates of progress ami mastery tlm'e. Om 
can also measure genera) achievement and changes of aptitudes. These 
measures are very useful in comparing the academic achievement pro- 
duced by IPI (o that produced by other systems. Doth Resnick (1967) 
and Undvall and Bolvin (1966) have predicted that under an individualized 
instructional system, students will show a highel- level of achievement 
compared with 'students studying under traditional instruction. This gain 
is attributed to the better match that this system achieves between the 
curriculum and the students' k-.owledge. needs, and interests. Never- 
theless, measures of academic achievement alone provide no information 
about the decree to which instructional procedures adjust to individual 
differences which are expressed in actual classroom behavior. 

, V In this study, it was first assumed that in an adaptive system of 
fducation. Individual differences along cognitive and noncognitive dimen- 
sions could freely develop. The . 't system ij not a full range "open" ' 
Hy.t.-m a.nc- children follow a prescribed order of learning units. Ncver- 
•hfless. the fact that children can complete the units" at their own pace, 
and the fact that they do not benave in a permanent teacher -controlled 
situation (as in the traditional classroom), m k.. iPl open enough to allow 
for individual differences to bo expressed. 

Stfcond. It was assumed that if individual differences are actually 
express^^d. ,t should be possible to detect them by. for example, obser- 
vational methods. These observational methods may show how different . 
I<inds of sl.idenls bfl.ave and how they int.-ract with their pe.-rs, teachers, 
and other faclurs in the educational settinK. 

rhe que.stion remains as to wnat aspects <!f behavior bevit indicate 
adaptability. jwo .issumptions about adaptability were made in this 
Htudy. rhr firM IS that adaptability of the curriculum to individCtal dif- 
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fcrences can be delected hy mea8urin,i the students' on-task/otf-task 
behavior. If the curriculum is well ma-tched tb individuardifferonccs 
(by the curriculum builder and by the clas.sroom teacher),' slow and fast 
sludcnla Bhould not diCf.-r in the amount of time they spend in on-task 
be. .-ior. be lihal prescribed, mtilually af-rofd upon with the teacher, or 
If- initiated ta.sk. 

The second assumption is that ndaptabilily of the teacher to^ndi- 
vidual needs can be detected by measurmn the number of teacher-student 
inleractif.ns. The circumstances under which these interactions take 
place (e.«.. on-task or off-task) is also impor'an'. In general, it is 
assumed that there should not be a difference in the aVa LJaJPiUiil 
teacher to either slow or fast students. Occasionally, however, when 
there i s^an appa rent need to compensate for a disabili y evidenced in the 
learning pr^ress .,f a slow student, it is expected that .he^-acher- will 
initiate encounters with the student in order to assist his learnm« efforts 
and to provide extra emrttional support and reinforcement. 

From the assun.pt. ons made above, it can be seen that characteris- 
tics of adaptability can be defined in terms of two dimensions: the task- 
rflaled dimens.on and the interpersonal dmionsion. 

Three possible kinds of task-related learninK v/eie defined: direct 
on^ask behavior, task-oriented behavior, and ofMa_sk behavior. Direct 
on-taTk behaviors included activities .n which the student seemed to be 
occupied by bis tasks (e. «; . reading, writinu. Rame playin,. counting). 
The task-or.ented cateuory included activities closely related to on-task 
behavirtr ihou^h „ol considered as dir.ect on-task behavior (e.K., waiting. 
arran«in« an assi^nn.ent ). S..ch activiti-s could indicate learning ten- 
dency or motivattun. Activ.ties in which no relation to task pe.-formance 
w.is apparent were included .n the off-task category. 
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The second behavioral climonsion specified the types of behavior 
engaged in by .he student while performing direct on-task! task-oriented, 
and off-task activities. Three types were defined: no interpersonal 
interactions, dyadic interactions (student- student, student-teacher), and 
U^rtni]} interactions. ' 



-n.ese two dimonsipns were combined into a matrix from which 
categories of observed beiiavior were derived (see Figure 1). Not all 
cells of the matrix were measured, however, since not all combinatLorfs' 
of behavior can occur under the IPl system. f 

■i'l}m.£ll- The subjects for this pilot study were eight second-grade 
atudenls from the Krick School, an inner-city school that is associated 
w.thahe Learning Research and Development Center. Four oRhe stu- ' 
dents had -mastered- the largest number ^ curricular unils^!^ J^^re th^-^^ 
classified as 'fast. " The other four had mastered the least numyPT^ 
units and were designafed as "slow. 

* 

Appara Uj^. In order to record the different amount of tlmo spent 
in each category of behavior, a graphical recording device with a remote 
control buttoli box was employed. This device enabletl the observer to 
position himself At any point in the claseroom. 

Procedure 

The observations in ihis pilot study took place in the winter of 
I'f ^ in onv of the H.-iond-grade classrooms in Friek School. ;rhere 
were Hi sludenm m ihis e la ss r.jom. Observations were taken during 
the nftjrning hm.rs (from "-.M) lo 11:30). During these periods, one 
teacher or aide "traveled" among the students tutoring those who 
reciue.aed help, whlT.. the other provided assistance to a student or 
gcowps of sludeniH-for more . xtended periods of time. Half of the 



•^ ■' ...-,...'(,,,,■ U.I I),. .„lu r half 

r<..i.lin«. II,.. n-x ■, ; - |. <• w' u vv-.rk .•,.„. ,,,,.1 1, work.-l „n . 
and «.c<- v<-r«.i. Cm,:. .1.. .n.ts w.r,. sl,.lv..,| .,^.,i„si th,- wal.s ..-u: , 
Ih- M.KlentH v.,ul,l ,n<l.-,.. n.l. n-U w.nlk t,. ihc hIh k',..s i.nri pick uplu-/!,, .• 
' itppropnaio ta.sk. 1 i„. ,!„.(. „t- xv.-rr -.{.so nwan- ..f how t<. arrant<<- li.. 
niaH-rinlf. for ;<m, h.,v. :.. rroi>;aMi/.r ih.-m aft.-r Ihiy w.-io thfn.^t 

ami how U) r< o,.. n: . , ih,.y w.t<- ..v..h..hl. lu otl., . s. 

i 

Only ..nr m . is l,.. rvffl at a U-ur. Ihuu^h t-ach stiidt nl v i 

obHcrvwl fiv*. ti.u.' . rl rfii. i uly. The pvvauM.^^^ h of rrtMr cnt 

between UiL' iM/c) obs.-r.i.rs wrr- checked by vicJ^tape rquip'iiifnl and v 
found 10 bo bcMwcc-n and 'M p*M-( .-nt.^ In ordt^r u> control for subjc-ct 
nialti-r efft^t. :>b...i. r .mi r)ns were made an rquaJ number uf linu's when • 
the student 8 wure s. mImml- math and reading. Similarly, the eff/ct of a 
particular leather 1 1 ho * on> rol led . (fwo p<.r.sonM/as ,t is explained 

above, wert' i-uidin.' . idmls-.the r.iain leacher and the aide. allluMi^h 
only fine was m contact with the .-student at a lime. ) 

* 

if i' 'A f i25? I 'l/* c < 1 . s H » u 

Thi- re-ilts pn-M-nlrft below cont. uM>tr t)n; (I) c ha ra i tt r i s t ir ^ 

of slow aufj f.isl siud-nls whi-n Uuy wt^-ked null vidua lly. (.^| i harac tens- 

tics of student -learher mleraction of slow and fast students, and (4) 

characteristics of thr pi er int.-racfions of slow and fa^l sludeiUs. 
e 

liL^^'Yi^iiL^U' 'JVi»jn. labl. s I and Jt show Ih.- ri-sults ,n thos.- ^at*- 
nones m which ihe individual fjoen lujt interact with eith.-r prrrs ui a 
teacher. On the average, ihc fast Hludeul spent twice as mu( h (in . 
working on his assmnnu-nt than the slow one. Th.. n.unbrV of iucurr- u/ 
of on task behav'ior. howevrr. was about ihr sarr,.'. Ihe prrcmfa^r oi 
tinu- spent in direct e)ff task behavior by slow studc^nts was twin- that 
spent by Iho fast ones. I^cfh thr slrwand the fast students sp^uil about 
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thr Hame amount of lime arranging assignnu'rils, Tho luimber of waiting 
occurr ence^ was .significai^ly higher for fast stuclentj?. 

As noted above, the fast sUulent worked more and idh'd less thiin 
the slow oni'. If our firsl af?mmption that curriculum adaptability c/in be 
detected .from rneasurvs of on-iaslj behavior is correct, this result indi- 
cates that the present curriculum in not equally adaplivi: to thr wnlirc 
range? of individual rliffe renci-s, The term curriculum is used here in 
its hroadi'st sen sr. 1; is not jutit tht^^^nstruct ional unit or a sequence of 
units, but also the kiml of decisiort-rnacU' by \hr teacher about presenting 
a Vpi c ifii curriculum unit- to a particular student. If it can be assumed 
•'hat thi' iraeh'^ r g?n)|Vially made the right decisions within the conlOxl of 
what couJd possiblVbi; done, cur r iculnm devi-iop*- rs should then recog- 
iU/.r,the limits of llu- pri'.srnt ( urricitlum to attract and hold tlie attention 
of different studi»nib. 

In any cas'-r it seems that it is not- thi* theory of the "proper? mevtch^' 
but rather the aetua 1' present attempt to approximati- that match with a 
wide i-anue of individual' differences which net'ds to br reconnldt?)*^^ 
Several rx-fisons van accotint for this situation: (1) ROnie students (slow 
/ oin's) require n\ori gradual sequencing; some pre requi/iites for 
achii'ving Ihr taskh may not be previously obtained; (3) differences in 
inlerests are not met by the present curriculum; and (4) available cur- 
ricu' m^ units art" not variJ?d i-nough to equally attract all students. 

Thi' fact that botii the slow and the fast sHidents spi-nt about thi- 
same pericnta<e of limi' m arranging assignnu-nts may indicate that 
this amount (abotil 14 piM'cent) is a characteristic of the iPI procedure 
. rather than a fliff^Tt-ncf between slow and fast studi nts. Interestingly, 
with rrgard to rhi> category, the standard deviation among the slow ' 
sHidi nts: wa«^ ' r Imies higher than that among the fast ones. It should 
be rrmenibrred Miat slow students have, in fact, less cause for sfji-nding 



timt- in arranging assignments since they master fewer units in a period 
of time. Therefore, it seems that being busy with arrangements may 
hhve jnorc than one purpose for some of the slow students. Perhaps 
n\akin«; ar rah^^ements becomes an end in itself or, perhaps', some of 
the slow students are hampered by the need to arrange their assignments\ 

With regard to "waiting" time, the difference found between slow 
and fast students was in the number of occurrences of waititTg behavior 
fbut no! [hv total amounTof time spent waiting)^ This difference is xmi^ 
surprising if one- remembers that fast students master more units, and\ 
that any pronrt'.ss to a new unit must be approved by the teacher. More- 
o\fr. (he fast students not only master more units, they also face more 
points of novelty and difficulty; theyj probably are less reluctant to request 
help from the teacher, and are morl; oriented toward academic achieve-^ 
menl. It thus $eems that- one should expect the fast students to exhibit 
more waiting behavior. ^ 

•iL^l^L^inl^Jl^lil£!l^Li[lLS^^^^ '^'he frequency of interactions for 

iiivinu i-encral attention was less than 1 p. nt in the behavior of ^both 
slow and fast students, and thus is not inclucit d in the tables. The irifre- 
'quent occurr.>nce of thi^s behavior may indicate that disciplinary problems 
were unconin .Mi in this particular tlassroom. This finding is in accord- 
anre with the.i^enera! belief that in il«H.^i!lUili*lJ_L'L« ti*.«c^ in-whijth 
every pupil is free to work at his own pace, there is a minimal qVed to \ 
control or to discipline students, 

Tfie number of occurrences of student - teacher interactions which 
cmu'tM-ned on-task activity was significantly higher with regard to the , 
f^^st stu dents comparted tn \hv slow oneji. (See Table 3. ) This finding ' 
is Apparently a result of the previously mfcintioned hi^^her fate of "waiting" 
frrqiu-neifs among fast students. If this analysis is corr-ct. it cart be 

; 
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saifi that teacher behavior in this classroom is in accordance with the 

principles of adaptive education, i.e., the teacher is responsive to the 

apparent needs for help, There are, however, two points of concern. 

First, if teacher behavior is a simple function of students* requests, 

, the teacher, then, does not initiate interactions with slow students who 
f 

may otherwise'nol request help. There is some evidence (Klein, Roden. 
^ Cientile, Kesnick. Heynolds. ^ Bachmeyer, 1972) that increasing the 
num^)er of interactions bet^^•een teachers and slow students results In 
higher oii-task behavior, ITiese interactions probably have an important 
rewafrd effect on the student, 

Sec jnd, as plight be otherwise predicted, there is no evidence 
from the data gathered in this study of teachers favoring particular 
kinds of students. The fact that fast students had significantly higher 
frequencies/Of teacher guidance can be explaim-d, as indicated abov/R, 
by teachers* reoporMivene ss lo the greater number of requests for help 
fr om the fast students. In the psycholoj^ical literature, some psycholo- 
Hists have asserted that favoritism does exist. For example, Rosenthal 
an<;l Jaccbson (19f,«) and (^ood and Brophy (1969) claimed that teachers' 
expt'ciations of student performance function as ^self-fulfilling prophecies, 
such that positive teacher expectations te>id to increase student perform- 
ance. Ciood and Brophy found in the traditional classroom "pro-active 
teacher behaC'ior that ^oes beyond the objective differences among the 
children and vSU^^est that teachers may be enhancing these differences 
^/^\./ySth<|- than reducing them through compensation technique^ " Again, 
^ even tho\i«li this study did hot concentrate on this particular question, 
the eviddnce ^^athered does not substantiate a claim of teachers^ favori- 
tism. I 

Student -Student Intera ctions, The remarkable finding in this 
domain is that occurrences w©re recorded in the "dialogue with;neighbor" 
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category in the off-task dimension (sc.- TabU; 4). but almost no |n-<T • ^ 
activity was recorded in the on-task categc^i ies C 'lrlp other sludc-nt" 
and "Riuded by student"). Apparently, the U'l sysle.n ha.s .lyl yet made 
prolislons (at least in chis particular classroom) for peer k-arnin,i and 
peer tutoring whereby sludents can help each oth. r or cooperate- in n 
common learning task. 

Nevertheless, both slow and fast students sp.'i>l about the same 
amount of time in dialogues of the off -task kind (between iaa« 16 percent 
of their lime). This is' not a negligible amount of time and might'indicate 
a real heed for social interaction, a neo'J that does not change drastically 
among academically motivated students. It might be helpful, there- 
fore, to channel some of these social interactions into the learning oo- 
task domains. 

G roup Activitie s,. The degree of social activity (see Tai.le 5) , 
which Involved more than simple dialogue was ver^ low for both fastt ^ 
and slow students. The total percentage of time spent in this, activity 
was about 1 percent. On thejiverage, the number of interactions that 
occurred in 20 minutes of observation was less than one, and^here 
were no significant differences between slow and fast students in th^s 
respect. There are two possible explanations. First, more social 
activity goes on in the classroom in the afternoon than in tho morning - 
when the observalions wer; made. Second' math and reading are the 
only subjects studied totally on an individual basis. , ^ 

I 

S^umj2iar^_and_Educa^t_iona^^^ 

It appears that'the observattonal system employed in this pilot 
study is sensitive to individual ^iifferences since the results clearly 
show two distinct patterns"of behavior for. slow and fast students. These 
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findings permit discussion about the way the educational system adapts 
itself to some aspects of individual differences. 

U IS often asked how to explain the slow progress of some students: 
j because the slowj^ent works slowly or because he simply works 

/ less ' A major finding of this study clearly indicates that much of the 

variance in the behavior of slow and fast students can be attributed to 
the fact that the slow students spend less of their time in on-tarik activi- 
ties. If this findin« is universally true, it should be faced by curriculum 
developers, instructional designers, and all others tor whom adaptive 
odtication is a major concern. The display of curriculum of adaptive 
education (see Kesnick, l97^) should vary in such a way that all kinds 
f>f students find it highly attractive. i 

In ihe classroom observed in this study, much of the control of 
the teacht»rs' time was left to the students themselves. In such a situa- 
tion, tlie fast students will always interact more often with the teacher. 
If adaptive education were to become compensatory for those who come 
to the scliool with obvious disadvantages, one would expect teachers to 
plan their activities in such a way that interactions with slow students 
will not depemi on student initiative alone. Perhaps cne way of achieving 
this would be to provide the type of data gathered in this study to teachers 
so that they can plan the distribution of their time in a manner which is 
more sensitive to individual needs. 

Since the findings of this study showed that the fast student worked 
longer, idled less, and r ceived the teachers* help more frequently, one 
ruay conclude that the system favors the fast student. Nonetheless,' it 
should be ri'inemberecl that the production of the fast student, in terms 
of thf number of units maslpred, is fouiul on the average to be three 
times higher than that of the slo^v ^tudont. Hence, if the total amount of 
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time spent in each catcKory is divided by ih4 number of unils lijastered 
durins Ihc period of observation, every difference found in the study will 
simply change lis direction. That is. when lotal amounl of ti^e spent 
in each catOKory is divided by the number of units mastered in this same 
li'»iie. the fa^t student appears to spend relatively less time in on-task 
behavior in terms of time per unit, and has the teacher's guidance less 
frequently. What emerges from this analysis is that in terms of educa- 
tional -cost. •• the slow student is much more costly to the system. Con- 
sequently, it miMht be said, that the IPl system (as well as many others) 
invests more In the education of the slow student than the fast one. 

And. fina'lly. the IPl system, as many other early education 
projects, has a generally high °adult-to- student ratio. Yet. even this 
hiKh ratio cannot satisfy the students' demand for teacher guidance. 
Hoth slow and fast students spend about 10 percent of their time waiting 
for the teacher. They spend more time waiting tha.i beiiig guided and 
the number of times they indicate waiting is persistently higher than the 
number of times they receive the teacher's attention. If there is no way 
to Increase the adult -to- student ratio, maybe alternatives ough^to b* 
considered.''^ one of which is for the students to help each other. Another 
way to solve the guiding problem is to increase self-controlled tasks 
(Wang. 1'173) which presumably require less teacher intervention fof 
some of ihv students. 
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When tht»y start school children frequently confront large numbers 
of pocT.s for the firKi iimo, Durmu a relatively short period of lime 
childrfn establish a poHilion for themselves within their classroom, 
make friends, and learn to cooperate with others. An elhological 
approach to the analysis of this process would suggest that the ways in 
which c hildren be^in integrating themselves into classroom social struc- 
tures should shojw phyloKonetic similarities with the social interactions 
'pc-rformed by the jlrveiules of other primati? species. These similarities 
could occur on the levels of: (1) gestures expressed toward peer«, (2) 
the amount and type of physicaLinteraction. (3) spacing patterns and 
activity levels, and (4) the total orj^anixation of group structures. 

In IraditL.nal etholo^ical studies, the research procedures would 
include makinu detailed records of particular behavior pattern^. These 
patterns would be described m terms of their physical attributes without 
any molivationa] causes bein^ ascribed to them. The total set of patterns 



The results reported in this study were supported in part by the 
United Slates Office of Education and the National Institute of Education 
The opinions exibressed are not to be taken as reflecting the view of either 
the Office of Education or the National Institute of Education. 



ini 



ERslC 



or behavioral charactoristics would conatUute an ••othogranV of the 
spccros being studied. ' A varioty of individ.ials within a species would 
bcinvostiuated. e.«.. youn,.. adult, male, female, and the portion of 
the otho,ram performed M the various individuals would be noted... 
Sequences c.f action and response between individuals wpuld alsp be 
invcstiKated and then used to generatp^ypotheses about the relation- . 
' ships between actibns and between the individuals performing them' 
(cf. Hlurton Jones, 1972). The basic methodology within this appr«adl 
is naturalistic observation and the fineness o£ detail is dependent on the 
• particular recording devices us^. e.g.. observing, filming.^ video- 
taping, etc. * 

The behavioral characteristics that emerge fron, ethological 
s.ud.es would be expected to constitute behavioral elements that, to 
some larger or smalWr extent, are: (1) universal to the species, 
(2) exhibit aex differences, and (3) exhibit individual differences resuft- 
■ ing fron. particular genetic constitutions and/or particular environmental 
experiences. Behavioral repertoires of species ^uld contain some ele- 
meats of bc-havior '|hat exhibit phylogenetic continuities, and some ele- 
...„ts that are species specific. The first r/ects adaptations that u ere 
necessary for survival across species encountering broad similarities 
in environmental situtations-. while the second reflects adaptations to ^ 
the species historically specific environment. For example, vocaliza- 
tions such as crying might be expected to occur throughout the rt.ammaha 
phylum as a signal Indicating that severe^nvironmental pressures are 
being experienced by the infant, and that he requires the mother's atten- 
tion. But crying as a component of.loneliness would not be expected to 
occur in those species which experience regular predator pressures and 
where the infant may be left alone for extended period, of time (e.g. . 
rabbits, deer. etc. ). Thus selective pressures will elicit or suppress 
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behaviors in species experiencing different environmental sitvations, 
i.uarntd or conditionetl behaviors are then modifications of this basic 
rc'perloire, but lhi?se behaviors are always limited by the possible set 
of behaviors ihai the organism is capable of performing, 

Tht\ same ethological methods and perspectives are used in the 
study of human bL-hivior, F'rom the Ii,terature on primates, the human 
ythologisl obtains chies about \vhat to look for. In the present study, 
the approach of proct-eding from data to determination of patterns to 
goni'ratiDn of hypotheses was reversed because of the extensive primate 
literature which exists. This literature indicates that the process of 
forming <jlominance hierarchies among peer groups may also occur in 
chil<irrn;! Children's concept of the social organization of their world 
was dt'termini'd by asking the children about it» First the children's 
(r^^c play activities were observed to determine if their group str\ . .s 
wert- similiar to those found in primates ^and, hence, could lead to hier- 
art'hical ;;ocial stnictiires which might be presumed to resemble those 
found in priiuate troops. 

Rationale 

. A basic frature of social structures in primate troops is the 
dominant i« hii^rarct./. This hierarchy is a linear or quasi-linear arrange- 
ment of individuals that provides differential access to portions of the 
environnuMit, it may also provide a locus of attention within^ which cer- 
tain individuals can f mction as troop leaders, and it definitely provides 
a group struclvire which, niiriimi/es inter- individual conflict. One of the 
problems fating \hv young as they grow and develop is to be able to 
establish and maintain a position within this hierarchy. 
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A similar problem appears to face^human children. For eventual 
functioning within adult society they would' need to first be able to estab- 
lish and maintain contact with their peers. For each child this involves 
being able to recognize relationships among peers, and making a place 
for himself within these interaction networks. Of perhaps greatest impor- 
tance in this process of moving toward others is for a child to be able to 
maintain a high feeling of self-esteem until his position within his social 
world is fairly secure. 

In this study, phylogenetic continuities were sought between pat- 
terns of social interaction in children and other primates. The interac- 
tions examined were then presumed to affect the children's perceptions of 
.their peers. In particular, the children were questioned to find out if *" 
dbminance hierarchies o/curred within their classes, and what the develop- 
mental changes were W these hierarc^hical structures. Four hundred 
fifty children from a private middle-class school were observed and 
tested. They ranged in age from nursery school through third grade. A 
variety of methods were used but because of time limitations only three 
will be described: (1) group size, (2) nearest neighbor data (collected on 
the playground), and (3) a hierarchy test (given in the classroom). 



Play^^round Observ ations | | . . 

A total of 435 children, 'enrolled in preschool through grade three 
of a single school, served as subjects for the present study. Two different 
observational methods were developed for the study, and the two observers 
employed for the study were specially trained to use these observation 
methods. 

The first method-.group size- -involved watching the playground 
for a few minutes to see which children were moving together in games 
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or other activities, and then scanning the playground from right to left 
recording all of ihv groups as they occurred. Children moving together, 
e.g.. as 'in a game of dodge ball, were coumted as being in a group, Chil- 
dren stationary, but within one meter of another child, were counted as 
bein^ together, even if they were not interacting. A child standing alone 
was counted as a group of s.ze one. 

The second methods-nearest nthghbor- - recorded individual 
spacing patterns and modes of interaction (Kummer, 1968; Sommer, 
1969). A scoring sheet was used and is included in Appendix E of this 
paper (see Criteria for Observations). In particular, the four categories 
of behavior that were recorded on the sheets were: (IJ the first, second, 
and third neighbor of the child being observed, (2) the relative distance 
^ between the child and each of his neighbors, (3) the type of interaction, if 
^ any, occurring between the child and his neighbors, and (4) the activity 
: level of Ihe child during a portion of the observation period. A first neigh- 
bor is defined as the child spatially closest to the child being observed, 
the second neighbor as second closest, etc. 

The observation period for each class was the morning recess 
period, 15 to /.O riiinutes in length. The classes tended to stagger their 
rvcess periods so that normally an entire class could be observed without 
tunfu.«ion. When ihe rJiildren were of more than one grade level the size 
> differential belwc i u ih.- children permitted recording to continue. In the 
ft w instances whi-n nuu e ifian one class of the same level were on the play- 
ground th' children were ri»corded as being in kindergarten, first grade, 
etc. . and not in a particular class. 

lo be awrv of observing each child, the number of boys and 
Liirlh in a class weri noted as they came onto the playground. Thn. child 
to be observed wa.s selected by dividing the playground into two halves. 

ins 
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A boy and a girl were observed on one half, and then a girl and a boy were 
observed on the other half. A short description was made of .each child 
so that he or she would not be observed twice. The child was selected, 
watched for approximately ten seconds to familiarize the observer with 
the Situation, and then the child's interactions, distance from neighbors, 
etc. . were recorded for the next IS-second period. The distance covered- 
the activity level of the child-was recorded during the subsequent 30. 
second period. ^' 

This present study Was an attempt t/relate childr,6n's group- 
ing tendencies to the action, performed by other ground-'dwelling primates 
The types of interaction which occurred on the playground are grouped 
categories of behavior rather than descriptions of finely detailed physi- 
cal movements'as found in an ethogram. .Th'e.e are two reasons for this 
procedure. F.^st. the broad. patterns, together ^yitl, the children's per- ^ 
celved hierarchies, would be sufficient to generate hypotheses ab^ both 
children's and primates' group behavior that could be examined in more 
detail later. Second, an ethogram is meant to be a nearly exhaustive 
portion of an organism's behavioral repertoire. Hence, the preparation 
of one is fairly time-consuming. Since the rules of data collection requir 
that the definitions of the behavior patterns be complete before data col- 
lection begins. 2 the present study served as a time for preparing the 



2 behind this "rule" is that the apparent first appear- 

ance of a nel behavior pattern in the middle ^ ^^^^^^^^ 
want to begin recording °"--n« ^e ^ showi g^^^ 

between groups, etc. 1" f*'^'' '"T " aware of it. Hence, 

plv took some time before the investigator became aware ot i • 

:udden mdusion might we.,h ^gai^.- a ^-p w^^^^ r^tr^thls ' 
but had not been noticed, bcorinc from flltned recora 
problem. 
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ethogram while collecting other data that could be immediately analyzed. 
The ethogram was then used on a subsequent study. The categories of 
behavior listed under "Criteria for Observations" (see Appendix E) were 
chosen to be broad enough to give a feelin-j of what occurs on the play- 
ground while still indicating areas where sex and age differences might 
be profitably explored with the finished ethogram. These later results 
are now in preparation. (For human ethograms already completed, the 
reader is referred to Blurton Jones, 1972, and McGrew, 1972. ) 

R esults of the playground observations. In the grades after 
nursery school, the boys' groups were larger than the girls' groups 
(Table 1). This was both in terms of the maximum and average Vnember- 
ship of the groups (p<.05). The average size of the groups increased 
for both sexes as they matured, but the boys' rate of increase was higheV; 
Thu maximum size of the boys' groups continued to increase across the 
grades, while the girls appeared to have a maximum of five or six chil- 
dren at any grade level. The very large boys' groups usually contained 
a few girls, and in those classes where the children were known to the 
observer those girls tended to be near the top of the girls* hierarchy. 

Table 2 shows that boys played with boys, while girls tended 
to be near girls. Boys were the three nearest neighbors of boys in 48.5 
to 74.0 percent of the observations; the girls, similarly, had girls as 
neiKhbors in 48. 5 to 69. 9 percent of the observations across the grades 
Trom im'i'sefy school to second grade. Girls were also near teachers for 
a sif(nificantly larger percent of the observations than were the boys 
(P <_ • 00^). Fof both sexes, contact with an adult declined steadily with 
increasing age; the largest decrease occurred after nursery school where, 
over 20 percent of the observations showed both sexes near a teacher. 
The hljJh percent of time nursery school children were near teacher s^tnay 
be Hue in part to their also beinj; observed insid*; during "free tirni^" 
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Physical interifction, such as playful wrestling, holding hands, 
or throwing a ball to one another, increases with age for both sexes, but^ 
» it reaches a peak for boy.s earlier than for girls (Table 3). Agressive 
and verbal encounters were recorded separately. Boys were found to be 
significantly more aggressive than girls across the grades (p <_ .009) 
with peaks in kindergarten and first grade, while girls were more verbal 
• than boys at all grades (p < .005). 

For all grades from nursery school through second grade the 
boys were more active (p < . 05). Boys also maintained a greater dis- 
tance betA^'een themselves and other children (p < . 05), and between 
themselves and any teachers present on the playground. If opposite sex 
neighbors happened to occur they were significantly farther apart than 
same sex neighbors (p <^ .001). , 

In general, the playground groups of young children were found 
. to be like the social interactions which occur among ground -dwelling 
primates. The boys gathered in larger and more active groups than did 
the girls. Boys played with boys, while girls played with girls. The 
girls also associated with adult females more frequently than did the' 
boys. The boys were more aggressive but, as with some juvenile female 
primates, some of the girls could be found in the rough-and-tumble boys" , 
groups (Kummer, 1968). 

Hierarchy Test 

The existence of dominance hierarchies in primate troops is 
generally inferred from the physical encounters which occur between 
troop members. The recording of aggressive encounters on the play- 
ground only indicated that boys fought more than girls. A hierarchy could 
not be derived from these observations because the individual children 
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involved were nol recorded. Instead of making more deUiled observa- 
lions, we questioned the children about the dominance hierarchy which 
they perceived within their classrooms. The basic question asked was, 
"Who is the toughest ' ""^ Two different forms of the test were used 
depending on the age of the children. 

In nursery school and kindergarten the children were photo- 
j^raphed and snapshot size pictures were made of each child. On the day 
of the test each child v as individually taken out into the hall. He was 
shown the photographs of his classmates, placed horizontally on a bench 
or on the floor. The photographs were arranged in alphabetical order 
by first name. 

The instructions were: "I'm going to ask you some questions 
about your classmates. The first question is about toughness. Now 
what is another word for tough'' " (If the child had trouble answering, 
he was told, "Do sopiething tough. ") Now let us look at the first child 
in the row. If that child is tougher than you, turn his picture over. " 
After the child acted, the experimenter made sure he understood the 
question. The experimenter then repeated the question with each pic 
ture until he was confident that the child understood the task. Then he 
said. "Now continue on down the row, turning over the picture of each 
child that is tougher than you. " 

From first grade through third grade a paper and pencil test 
was used. The children's names were listed alphabetically down the 
side of a sheet of paper. The paper was cut so that tabs with each child's 



^Other questions were; "Who is the ^smartest, nicest, and 
has the most friends'' " Of these, "toughest" had the earliest and highest 
dyadic agreement. Research is continuing in this area. 
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name could be easily torn ofl. The children were also given a sheet o£ 
paper with the numbers one through the size of the class listed down the 
left-hand side. The children were told to "Look through the list of names 
and tear off the name of the child who is the very toughest in the class. 
Place ihai name beside the number one. Now look through the list and 
find Ihd next toughest child and place that name beside the number two, " 
etc. 

Hesims of the hierarchy tests. The distribution of children 
across a dominance hierarchy results in boys being placed near the top, 
girls near the bottom, and considerable overlap in the middle (Table 4). 
« '^iis is the same pattern found iii' many primate troops (Carpenter, 1964). 
As can be seen in Table 4. the configurations of the hierarchies change 
with age., 

^ Tile basic unit of analysis for the hierarchy test was the dyad 

of established dominance. The percv^nt of dyadic agreement was formed 
by taking those pairs when; both chiklren agreed on who was the dominant 
child and dividing by the possible dyads from the class. For example, if 
John said that Bill was "tougher" than John, a^nd Bill said that Bill was 
"tougher * than John, this pair was said to be an established dyad on 
"loujtihncss. 

Followinu the ages suggested by Piaget, it was hypothesised 
that the school age child {age 1. first grado in our sample) could readily 
perceive a hierarchical relationship and, h» nee, wpuld have a higher 
dyadic agreement than the preschool child. This was tested on the kin- 
dergarten through third grade sample. In examining all of the possible 
dyads within each class it was found that there was a highly significant 
Hnt'ar trend (F r 79.2. p £ . 0001). The largest jump in agreement on ^ 
relative status occurred between kindergarten (40 percent) and first 



ERLC 



113 



03 CO 
O) 00 



U> 

ro 



ro 



(a> 00 



4^ 

ro to 



to 



O CD 



O 00 



^ 0> 



=• o 



ro 
u> ro 



ro 



(a) 

00 



ro 



CO 



CO 



: M 1 



ro OI (Ji CJ 



ro -* ro 
ro ro I CJ CT 



ro ro 
O) CJ 



^ ^ ro ro 



<5 



CJ 



a 

<9 



X 



D 
c 

5' 



7) 

0) 



CJ 



aiaviivAv Adoo iS3a 



"KT copy mum 



Mradv (62 percent). Tins large increase in percent of dyadic agreement 
corresponded to a similar increaije on a smaller sample where the under- 
lyinu cognitive level was measured (see Edelman, 1973). Children 
. seemed to develop a consistent perception of their dominance structure * 
at the same time as they were developing the logical operations of seria-' 
tion and transilivity» 

The peak in aji-^rcfssive encounters on the playground for ^he 
boys extended across kindergarten to first grade, and the children's per- 
ceptions of dominance relationships appeared to follow from this experi- 
ence. As discussed elsewhere, the younger nursery school children 
ftnij^bl t?iuch less and were almost unable to form a hierarchy (Edelman & 
Oniark, P>73a), lhe> ^ave extremely self-centered responses, almost 
always placing themselvt s first or secund in any hierarchy, buf (Experi- 
ences in kindergarten and first ^i;ade seeme<l to structure their percep- 
tions in a way that substantiated the view that a dominance hicrar by 
existed within each classroom, 

From the results of the primate studies, it was hypothesised 
that most agreement on "toughest" wouid be boy-girl pairs followed by 
bny boy pairs and then izirl-i^irl pairs. This hypothesis was clearly 
confirmed by thi» restilts. Moy-girl pairs had an agreement of 69,4 per- 
cent, lioy boy pairs had an agreement of S4. percent, and girl-girl 
pairs had ^1.7 percent. The difference between cross - sex dyads and 
boy dyads was siunificant (F 16»S, p <^ .0008), as well as the differ- 
ence between bov-boy and i^irl-girl dya<»s (F ^ 5.02, p _< . 036). 

Accuracy 

On the playgrouml. boys were foun<i to play with boys, and 

i^irls played with girls; Ihe boys were also in larger groups and more 
0 
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frequently involved in rough-and-tumble play, while the girls quietly 
talked in groups of twos and threes. Hence, the boys were seen as tougher 
than the girls, and they appeared to have worked out the status relation- 
ships among each other more completely. Considering the aniount of 
involvert^'ent within same sex groups, it was rather surprising to find- 
that each sex could produce the same hierarchy for the opposite sex as 
that sex produced for itself. • 

The children were scored for their accuracy of perception. 
The percent accuracy of perception measured each child's perception of ^ 
dominance in those pairs of establisb'ed dominance of which he or she " 
was not a member (number' of corr.fect choices of the dominant child in 
the dyad of established dominance divided by the total nilmbef of dyads 
where agreement occurred). Although .there were differences in dyadic 
agreement aciohg the different sex pairings there Was no difference in 
the percent accuracy of perception for these sex pairings (F ^1. 17). 

Not only did boys and girls have a similar level of agreement 
about their' own sex group, but the rank orders produced by each of the 
sex groups were highly correlated. The average correlation in a class 
between the average rank orders produced by the boys and the order pro- 
duced by the girls when ranking the boys was 0.86 and when both groups 
ranked the girls the average correlation was 0. 79. 

Therefore, although the males were more involved in working 
out their dominance relationships with each other and. hence., have a 
more clearly defined dominance order, the girls can perceive the domi- 
nance relations of both boys and girls as accurately as can the boys. 
This fmdmg supports the parallel to primate social structure suggested 
by Chance and .lolly {\'K0) that stable group functioning is dependent upon 
all numbers of the group paying attention to the dominant members." 
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Overrat in^ 

Enlranco into a dominance hierarchy would not appear to be an 
easy task. As s«en earlier, with children, or at least boys, this means 
involvomonl in ajiKressivo encounters and, at best, numerous knocks to 
one's. eKO, The egoccnJ^rjiit^y-ofTiur sury school children suggests a 
moans througli which this c; trance may prove tole less than traumatic. 
A, found by MCC.rew (1972) in his study of prc^ohoolcrs, it 'did not seem 
tu-niAtter in terms of children's subsequent behavior whether they lost 
an.encuntor to another child of lower status than themselves. A simi- 
lar rtsiill seemed to occur, at least for the boys, in this study. 

Children of all a*ges agree'on their relative status relation- 
ships with many of their peers, and their agreement increases with age--, 
from virtually no aftreemont beyond the bcy-girl difference in nursery 
school£to 66 percent agreement in third grade. Of the remaining pairs, * 
both children could say th«t the other child is tougher, 0r each of them 
touJd sav that the self was tougher. This latter was termed overrating 
and a porc-nt of overrating was formed by taking all of those pa^rs and 
dividing by all of the pairs in which disagreement occurred. Tabic 5 
shows the results. Hoys wore found to overrate themselves compared 
to othtT boys significantlv more than girls overrated themselves com- 
pared to other girls (p <_ . 006). Eighty to 90 percent of the boys' pairs 
whff disagreement occurred overrated themselves from kindergarten 
through third i-rade. In contrast, the girls' amount of overrating changed 
markedly during this period (K ^ 98 percent, first 28 percent, second = 
S9 ptMcePl. an^hird «l percent). 

For both sexes, the high amount of overrating and low rate of 
a«r<'enu-nt between pairs in the preschool years reflects the children's 
ogoceiitrism. While becoming more accurate in their perceptions of 
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.self ami othtTs as ihry ^row olck^r, the buys inainUin an oluinent of cgo- 
tiMilrihUK If ihurc- IS donht about a rola t lunsh ip, it is fl^cidi'd in favor 
of thf srlf, I he bov >, .onir.mi duMr peers and if ihci-f indoed is a domi- 
nance hierarchy, then ihis f>\ it ral injt; would appear to be an important 
pi rcepttial characn risiic for them hi-causc many losses may not matter 
if ihe other is seen as bemji; lower in the hierarchy. 

(Hrls t-n^ajte unly a few others at a time and their hierarchy 
is not as well defined. Their shift from overratlnj^ to underrating in 
firsl urade may be their response to adult authority, i.e. , to be "good,'' 
as well as a wiihdrawa. from the rough-and-tumble worlc^ of the boys-- 
a lime for waichin^ and learnin^^ about others. 

From second to third j^rade, both sexes also exhibit tendencies 
to overrate. Alihou,.h ihis increase was not previously hypothesized, 
some basis for explaining the increase can be found in the wo*'k of Piaget. 
In his study of moral judgment. Piaget (1965) differentiated between two 
attitudes toward following rules. At ages six and seven, the child Regards 
the rules of the ^ame as fixed and eternal. At a later age. the child 
believes that jjules are morV flexible and that he may influence how they 
might «et changed. In the case of making hierarchies, the first and 
second gradt^ children were trying to report the "correct hierarchy, " 
but in third grade the child may have been aware that he or she was 
making the hierarchy and could put people anywhere he or she might 
wish, When pilot testing the hierarchy test, we found^that the younger 
children asked if they were "right" after they formed the hierarchy, but 
the older children did not ask if they were right. Sometimes the third 
graders would put themselves high in the hierarchy and smile impishly, 
as though they just wanted to see therjiselves high. 
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C onclusions 

some of the social actions of children fit nicely within a primate 
paradigm. This is most evident with sex differences in the size of the 
children's groups, their amount of association with an adult, and the 
levels of activity and aggressive actions. The children's perceptions of^^ 
dominance hierarchies within their classrooms resemble the hierarC^^. 
derived from the actions of primates. ^ ' 

The boys were found to have more agreemc .t op 'th^^'ortion 
of the 4.i.rarchy than the girls, but that boys axe tougher than^girls had . 

^ tr,^ hnth sexes/' Ih #l>ite of the apparent 
even significantly more agreement for both sexes^^ m wpi 

lack of contact between the sexes during play.l^o^ds. the children's 
direction of attention included the opposite s.^f « was found that each 
sex could accurately perceive the hi^^arci^y created by the opposite sex. 
Boys were rr.ore involved in agg.e^.ive'^ncounters; an apparently neces- 
sary corollary of this was that.the boys overrated their own status posi- 
tion. Girls were not as involved in forming hierarchies and .howed a 
period during which the^'onderrated themselves. 

We are c-ontinuing research .n the area of the relationship 
• between cog^iU^e development and children's social experiences. Both 
th. ability to seriate and to pTrceive transitive relations would appear to 
, ^,ce.ve reinforcement from social encounters. If so. then the real world 
social interactions with peers might be a very necessary part of each 
child's educational experience. 

From an ethological perspective the interplay between cogni- 
Hve development and social e.^i.^cin, would be expected to have phylo 
,enetic as well as individually historic components. Humans and other 
primates mu.t interact in a social world so that one would expect adapta- 
tions for adequate survival in relation .to that world to occur through 
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nalural selection. In essuncc, this means that throughout a child's, 
flevolopment. including his cognitive development^ he would be expected 
lo exhibit characteristic behavior and thought patterns that enable him to 
fund ion adequately in the social world with which he interacts at any 
parficular age. 

The way children conceptualize their world at each age level 
can then be seen not only in how they manipulate slicks, volumes, or 
masses, but also in the way they interact with and perceive their social 
world. [he amount of agreement about one aspect of the children's peer 
groups ilml was found at a very young age indicates that the child's ability 
to seriate appears as early for social interactions as it does for the 
physical world. Another portion of the study indicates that this also 
occurs for transitive relations, and that a great deal of emotional involve- 
ment is attached to perceptions of social interactions (Edelman & Omark, 
1973b). Agreement about the structure of the social world occurred 
where <lirect interaction was very limited, as in the low Irvel of inter- 
action found between boys and girls on the playground. Children's abili- 
ties to pay attention to others is viewed as a very necessary adaptation 
that enables them to organize and develop their cognitive abilities. The 
age changes that occur in these abilities are seen as adaptations that 
enable the children to continue to move toward peers without being 
swamped by inforn?ation and experiences that they are not yet able to 
assimilate. 

If ethology lias a message that is relevant for child develop- 
ment. it is that a broader perspective needs lo be taken when viewing 
development. This perspective nuist include what the child brings to 
each encounter wilh his environment from his phylogenetic past. The 
question always posed by a researcher from the cthological approach is: 
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.-Why doea this child act the way he does? " We find it necessary to 
investigate more than immediate cause-and-effect relationships to find 
answers to this question. 
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DISCUSSION 
by 

Donald M. Medley 

School o£ Education 
UnivKVsity o£ Virginia 



The papers presented at this session exemplify a most encour- 
aging trend in research in teaching and learning, and testify both to the 
health and the potential of that trend. The trend I have in mind may be 
regarded as an increasing emphasis on means (or ^jrocess) as a focus of 
educational research in contrast to ends (or products). We are increas- 
ingly interested In "how" rather than "how well" teaching succeeds. 

After all. the education enterprise exists solely to maximize the 
effectiveness of the educational experiences the individual child has in 
school. We all learned in our first education courses that learning 
results from the activity of the learner. If this is true, then it would 
seem obvious that any successful effort to understand the educational 
process must concentrate on what happens to kids in schools while they 
ar? being educated. As one who has always taken this for granted, I 
find it difficult to understand why so little attention is paid to pupil 
behavior in so many research studies. 

In these five papers, we have five excellent examples of people 
looking for answers where the answers can be found. I am not surprised 
to find Omark and Edelman doing this--they come from a discipline that 
has characteristically studied phenomena where they happen, with a 
directness that looks almost naive. It would be great for educational 
research if more of us would take this kind of open-minded look at what 
children are doing, instead of trying to prove some pet theoty of our own, 
as we usually do. 
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The rest of our contributors also exemplify what I mean in vari- 
ous ways and varying degrees. Each of them shares an interest in what 
19 happening to pupils. The fact that each has some definite questipn to 
ask about what the pupils are doing does not seem to prevent their seeing 
what is actually going on. lliis idea of monitoring ^he implementation of 
an educational innovation at the point at which it is supposed to take effect 
(in pupil behavior) is exactly what I am advocating. For half a century, 
the great majority of attempts to change education for the better that 
were proposed and attempted were evaluated solely in terms of outcomes. 
Studies that collected data about whether there was any change in what 
the pupils did were the rare exceptions. There is no way of assessing 
the number of promising innovations introduced during these years that 
were abandoned as ineffective, not because they did not work but because 
they were never really tried. 

Lcinhardt's study is almost a textbook example of how a variety 
of techniques already available to anyone who knows enough to use them 
can be employed effectively to study the implementation process itself. 
The study casts li^ht on problems related to the fact that any innovation 
has different effects in different settings, with different teachers, and 
with different pupils. Shimron focuses more closely on a single aspect 
of process that is indeed crucial in the innovation he is studying; and 
the effort pays off. Wang's paper illustrates the role an ongoing examina- 
tion of learner behaviors can play In formative evaluation, enabling the 
innovator to shape the implementation In process. Ross and Zimiles 
are concerned with the important process of refining an instrument to 
increase Us effectiveness for purposes like these. 

7*hese papers share a primary dependence on structured obser- 
vation instruments, although we hav .;een, particularly in the work of 
Wang and that of Leinhardt, useful applications of other techniques. 
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Structured instruments like the one Koss Vnd Zimiles arc developing 
have unique a.lvanta.es that usually make them preferable to alternative, • 
procedures. Ihe precise, detailed, and objective information they pro- 
vide about behavior is their principal strength. Even when designed to 
test specific hypotheses, as was the case in most of those studies, obser- 
- va.ional data may reveal things not suspected by the researcher becauso 
of the quality of the data they yield. In this way. some of the advant.„e» 
of the openness of the ethologist may accrue-especially with a more 
carefully refined instrument like the one Hoss and X.imiles are working 
toward. 

There are two further developments 1 would hope to see before 
too lon«. To be content with analyr.in^ raw frequencies or percents of 
certain types of events- -however intuitively satisfying it may be-is to 
stop far short of full exploitation of the data. The data also constitute 
the raw material for a search for structure in human behavior-for / 
efforts to describe it in te?ms of fewer and more powerful, meaningful / 
dimensions thast are composites of raw frequencies. These are the 
stuff that theories ar^? made on. If human behavior is indeed lawful. = 
we should constantly be seeking for such laws, and they are to be fou.ul 
in the uniformities, the commnnalities in the detailed items of our / 
behavior rccorfls. 

The second development 1 look for has to do with reali/.ing the 
basic purpose of research, which is to enable a large number of ^-ople 

reap the benefits of the researcher's experiences. How effectively 
.bis is done depends on how accurately it is possible to con.municate 
i.st what happened in a study. Structured observational instruments 
are effective research tools mainly because they facilitate more accurat 
communication of what happens than alternative procedures do. This 
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results from the greater clarity the operational definitions of categories 
nuist possess--in other words, the greater clarity of the language in 
which we describe phenomena! Unfortunately, each of us, when he con- 
structs his owti instrument, is inventing his own language and vocabu- 
lary: and he i^, therefore, to a disturbing extent, talking to himself. 
Kvery new report requires lhi» ri^ader to learn a riew vocabulary. 1 
look forward to the day wlien have a amall nunjiber of standardized 
observation instrumt^rits so that the behavior of pupils observed in many 
diffcTont st^ltin^s and studios can br described objectivt-ly in comparable 
terms. Then our knowledge may bc-gin to grow in geometric rather than 
arithmetic incremrnls. 
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CO* • DISCUSSION 

by 

Lauren B. Resnick 

Learning Research and Development Center 
University of Pittsburgh 

The papers in this sy^nposium use observational technique, for 
several different purposes in the evi^luation and analysis of educational 
environments. Tl^ey also concomitantly represent the use of somewhat 
different methodologies. I would like to begin my discussion with a con- 
sideration of the different purposes of the papers in this .ymposlum, 
and then consider some similarities and differences in their methodolo- 
gies that seem to be a reflection of these purposes. I will conclude by 
considering some general questions concerning the role of observation.1 
methods in educational research-questions stimulated by the present 
papers but not specific to them. 

-n^ree of the papers, those by Ross and ZimUes, by Leinhardt. 
and by Wang, are explicitly concerned with some aspect of the e^luStlan 
. of educational programs. In each case the author, are evaluating their 
own programs, or those of close colleague, whose theorle. o£ education 
they share. The three studies fall along a continuum that correspond, 
roughly to the degree of control exercised by the development team ov.r 
the educational program under study. I^lnhardf. paper is concerned 
with the evaluation of a program as implemented in a wide variety of 
school setting., settings over which the developer, of the program do 
not have specific control. We might call her. a field evaluation .tudy. 
This concern with evaluation of the program a. It operate, under rela- 
tively uncontrolled field condition, sets limitations on her methodology. 
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but at Ihe same time allows a certain breadth of conclusion from h<jr 
data thai is not possible from sludit-s that are made in fewer or more 
contrcilled settings. 

Kqss and /.miiles's work represents a move from broad field- 
evaluation to detaijed study of three particular educational environments. 
The y cumpare their own program with a competing or "control" program; 
but s.nt t. ihoy examine two different settings in which their own program 
is used, they ar" not concerned strictly with the program's implementa- 
lion in a «ivt>n site. Neverlhelesr , they invest heavily in detailed obser- 
vaiM»n uf a ftnv classrooms, as contrasted with Leinhardt's much more 
U^ nt.rali/.ed ^i^servation of a larger number of classrooms. Compared 
.o I.i inhardl's approach. Koss and /.imiles's work is more expensive 
and nuich more likt ly to «ive us detailed information on the way m which 
iliildrcn behave in a particular environment, but less likely in its present 
[orin t,i provide an economically viable instrument for large scale evalu- 
at Kiti. 

Warm's study takes a still further step toward detailed analysis 
of a specific de\t'l(>per-c6nirolled environment. It is concerned with 
descriplimi and. therefore, evaluation of a specific aspect of a program. 
The effort depends upon deliberate modification of the program within a 
single classroom in the course of a school year. Like Ross and Zimiles. 
Wang studies a broad range of outcomes and, like them, she accepts as 
a constraint the i ifflculties of observing in a real setting rather than a 
contrived or laboratory setting. Nevertheless, Wang introduces an ele- 
ment of experimental control that neither Leinhardt nor Ross and Zimiles 
attempt. rhis is accomplished by use of a ^'reversal design, " in which 
a classroom is first studied under normal or ♦'baseline*' conditions; a 
new program is then instituted, followed by a return to baseline and a 
return again to the treatment conditions. This strategy avoids the 
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problems of comparing one classroom to another, wi^h the attendant popu- 
lalion'variat.ons. .Rather it atipws comparison of one .el of classroom 
con_clUions-wi.h another .set implen.ented by the same teacher, with iho , 
same children. Such a study avoids th. difficulties of control group com- 
parisons; on the other hand, without extensive replication, it allows ' ' 
,o draw few conclusions about the way in which the special treatment under. 
study--in th.s case, .he self- «cheduling system-rm«ht work in settings 

t 

other than the single classroom studied. • 

Sh.mron's and Omark and Edelmaa's studies are .More descrip- 
tive and explanatory than .^valuational in intent. Shimron.'s study takes 
on even smaller range o( behavior, in terms of both sample size and 
variables sn.died. than Wan,.'s. It focuses on individual differences Ln . 
the behavior of a sn.all number of children and in the teacher^s response 
,o .hose ch.ldren. all within a relatively stable classroom environment. 
Shimron considers -he teacher part of the general environment wLthxn 
which particular kinds of child behaviors take place. He looks for the 
pauerning .n these behaviors, .nd in this respect his work shares ,omc 
features with Omark and Edelman's. However, having detected behav- 
ioral patterns. Shimron seeks to relate them to academic performance; 
in this respect. is closest to Leinhard^. who seeks explicitly to meas-^ 
ure classroom behaviors that mediate success in academic learning. 

omark and Edelman's paper, unlike the others in this symposium, 
is no. concerned with .nstruction. but with purely social aspects of behav- 
ior .n school. The.r work dr..ws upon a tradition of research that is only 
now becoming fanUliar to psychologists and educators. The ethniogical 
tradition of observation, from which Omark and Edelman's work derives, 
i. based .n biological studi.. of animal behavior. As a .u-thod for study 
.ng chUdren. it brings .o Uu- study of human behavior some of the b.olo- 
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Bist's concern for detecting patterns of behavior, finding out what chil- 
dren are like in specific environments. 

Because of its different substantive focus. Omark and Edelman's 
work is of primarily methodological interest in the context of this sym- 
posium. Their paper does not describe in detail the methodological 
strategies, but instead focuses on specific theories, drawn from animal 
observation and now applied to children on the playground and in the 
classroom. Lying behind these theories, however, is a body of exten- " 
sivo observational technique and literature. As mentioned in their paper 
observation typically begins with development of an "ethogram. " a 
detailed description of the sequence of behaviors engaged in by an indi- 
v.dual. As repetitive patterns emerge, instruments for systematic ol^ser 
vations of particular behaviors are developed. The use of ethological 
approaches to observation of children's behavior is well described in 
recent books by Hutt (I970) and Blurton Jones (1972). The important 
nu-ssagi of this line of investigation for systematic observation of educa- 
tional processes is the stress on d etecti ng patterns of behavior, on 
treating early stages of observational work is a time for searching out 
patterns of behavior rathei than for testing formal hypotheses. 

The methods of observation used by the authors in these five 
studies are matched in significant ways to their purposes. Perhaps the 
most gt-neral point to be made in this respect is that whatever method is 
chosen, there is a •■frade-off" of some kind to be made; one gains certain 
kinds of information at the expense of other kinds. There is no perfect 
obst rvational system. Observation is expensive and one therefore 
seeks to maximize the kind of information useful for one's purpose in 
defigning and using an observational system, always recognizing that 
certain other kinds of informarton'w'ill inevitably bc-lost. 
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The result of this i,s that none of the studies reported here today, 
and I would venture to say that no single study likely to be produced in 
the coming y .rs. represents a perfect or complete examination of 
school processes. Yet. each offers something new. either substantively 
or methodologically, to our understanding of classroom processes. 
What we can reasonably expect over time is that a combination of dif- 
ferent approaches, used for different purposes, in different places and 
at different ttm^s. will gradually allow us to converge on some under- 
standing of what the process of educational interaction is like. My com- 
ments on specific studies must be understood in this light. They are 
offered not so n.uch to criticise the present studies for their lack of 
particular types of .nformation, but rather as a nieans of pointing out 
what each offers and what, in order to offer that, it has had to give up. , 

A general problem inherent in observational study of classroom 
behavior is that of cost: this cost needs to be measured in terms of: 
,a. time needed for the observation, to be conducted, (b) reliability both 
between observers and over tim^ and (c) difficulty of interpretation of . 
the d,ta ga^herecf. In gener^ I think it is safe to observe that the more 
detauVd the observation/and the wider the range ofphenomena they 
attempt to describeyaccount for, the greater the degree to which they 
encounter such cZ. Thus. Wang and Ross and Zimiles encounter 
.heater difficulties in establishing reliabilities and in reducing the com- 
plex d^iaji^iiccumulate than does Shimron; but Shimron's high relia- 
bili.Cand temporally extensive information on indiviAal children 
depend on limiting both the range of behaviors examined and the number 
of children observed. Leinhardt, in an effort to keep costs of the vari- 
ous kinds mentioned within reasonable bounds for purposes of field evalu 
ation uses ".ndex measures*' to a large degree rather than^ect obser- 
vations of the behaviors she is interested in. For example, ^e is 
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inlcicstw! tn^hi' d.-yrc-c k, which assignments are individualized. To 
study this qmrstton in d. iail would require cloOely following the behavior' 
of individual childrL-n ov.m- an <-xtt>nded pi-riod of time.' She cannot afford 
this kind of obsiTvnlion and uses instecvd a count of how many different 
• assi^jninenls aro Kiven in a particular time period. -To that extent the 
pro«ran, ,.s huini. individualized. While the assuriiption th^it this meas- 
ures individnalizalion is not directly tested., it has^face validity; and it 
is -imlirectly tested lat^by the ektent to which" it firedicts achievement. 

Certain details of the present pa'pers are useful .in raising [orj 
us some general issues concerning the use and interpretation of obser- 
vational Data. , Only a few cap be considered in the time available, so 1 
shall fommont here on issues that seem to me to be of particular inter- 
est. Wane's paper^includes in its title, and in its opening paragraph, 
reference fo formativf; evaluation- -that is. the use of observational 
techniques for providing information to the developer necessary for 
injprovementvof a program. Yet the paper itself gives us no detailed 
examples of how her data were ustd in modifying the program studied. 
It would be us^'ful for such descriptions to 1^ made available in the 
fuUire. Further, now that th^elf- scheduling program that she describes 
has been shown to be a rea^sonaVy effective one in the classroom, per- 
haps this suggests that the^^;X?fcriptive versus exploratory distinction 
cat^be (jlropped and that one should begin to focus on the quality of inter- 
action, (regardless of the label given to the activity. Wang's study, in 
other wjords. marks a'starting point in the study of an educational model; 
it is injthat sense properly labeled "formative. " I will be interested to 
learn the directions in which thjse investigations lead and to see docu- 
mentec, at least informally, the way in which observational research 
and program development mutually interact. 
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In the Wan, and Ho.s and /imLles studL^-s. ratings rather than 
direct observations are son.etin.es used as a way of reducing complex 
.vents to a measurable form. It is perhaps appropriate to consider what 
ratings are and are not useful for in observational work. Ratings are 
useful, assuming inter-observer .reliabiUty is adequate, when one has, 
dimensions already available and wants to kno^. in a rough way. whether 
we are approximating our own goals. Thus. tKey are primarily a d.^ic 
fo.. -su,nmat.ve evaluation. " i. e. . for decid.ng to what degree imple 
n,ented programs meet their goals. Rating/are generally not so useful 
when one is sea_rch_in^for important dimen^sions along which behavior 
varies ThcTy'do not allow one to examine relatively "raw" d^ta in search 
of patterns-, to formulate rather than Vest hypotheses. Gwei the present 
pauc.ty of knowledge concerning cla/sroom processes, a case can be 
„.ade for relying as heavily as po/sible on direct observation rather 
than ratings, at least at the early stages of any particular set of invcsti- 
^atiop^^. 

Hoss and /imiles make no attempt at all to relate the classroom 
process variables observed to the quality of learning as measured i.%de- 
pendently of the observational effort. This raises for me the interesting 
' question of whether process in the classroom can be equated with learn- 
ing outcome. What would happen if children from Ross and Zimxles's 
»radit.onal public school were suddenly placed in the Bank Street Follow 
-n.rough classroom ' How long would it take for them to behave like the 
rollow Through children . And conversely, how long would it take for 
,he Follow Through children to behave like the traditional classroom 
children, if their position were to be reversed. This question is impor- 
tant because it suggests the possibility that the behavior of children may 
be so heavUy mfluenced by immediate environmental conditions as not 
.o refl.ct a stable indiv.dual pattern at all. Rather, what Rois and 
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Ziniiles may bv measurinii is the extent to which they and their colleagues 
hjve succeeded in crealiiij^ the kind of environment they want to create. 
This IS an important step in the study of one's educational program, but 
it is only a step. The next step necessary is to consider whether the 
proi;ram, once created, does indeed have the effects one is seeking, 1 
do not mean to imply that programs should be judged solely on the basis 
of whether they promote higher achievement scores in mathematics or 
readint*. What I do mean to imply is that some measurement of learning 
effects that are independent of specific learning environments may need 
to be sought. This implies a "next stage" of classroom observation, in 
which individual children are observed in different environments and 
ovt-r extended periods of time. In such a program of research, Ross 
and /iniih's^s work would be viewed as an important first step. 

The richness of the data in these studies makes it difficult to do 
jusiicr in these brief remarks to the research reported. What is per- 
haps most important to note in conclusion is that this symposium and 
the papers in it mark a turn toward a new interest in description and 
analysis of tl}i»- educational process. This is a "new look" in educational 
research ehat, while tentative for the moment in the conclusions it per- 
mits, is of potentially major significance for increasing our understanding 
of and skill in improving the educational enterprise. The five studies 
repre sented here, while varving in purpose and method, share a com- 
mon concerxi.for describing how people- -c.ildren and teachers- -actually 
behave in school. This is a question about which we now have astonish- 
itil^ly little information, havin^^ focused our research over the past 
several decades vt ry heavily on academic outcomes. It is only by 
including in our study uf eclucation descriptions of educational processes-- 
as outcomes of interest in their own right or as mediators of more tra- 
ditional outcomes- -that we can expect to make sense of the educational 
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enterprise, to understand how. as well as whether, instructional pro- 
orams work. This symposium represents a most welcome movement 
in that direction. 
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Individualized Instructional Programs 

by 

Margaret C. Wang 

Learning Hesearch and Development Center 
University of Pittsburgh 
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APPENDIX A 
Thn LRDC Indiviriualized Instruc t ional Profi ranis 

The LRDC IndividuaH..ccl Instructional Programs include the Indivi- 
dually Proscribed Instructional (IPI) Program for children ..f elemen.ary 
grades (gr.de one through six), and the Primary Education ProjeC (PEP) 
designed for ch.ldren of early ch.ldhood age (age three through seven). 
Roth. IPI and PEP were developed to prov.de educational experiences 
,hat are adaptive to the learning needs of the ind.vidual student. The 
programs were designed with the basic assumptions that: (1) ch.ldren 
display a wide rang, of differences in their entering abilities and the ways 
in which they learn and acquire competencies, and (2) to provide educa- 
tional experiences that are adaptive to the individual differences moans, 
to provide learning situations (e.g.. classroom organisation, learning 
materials, etc. . that can accommodate .he needs of the individual student 
a^d when needed, .each the prerequisite abilities demanded by the learning 
situations (Glascr. 1972L 

The LRDC Individualised Instructional Programs are designed with 
the following guidelines (Glaser. 1970): (I) the goals of learning are 
specified in terms of observable student performance and the conditions 
under which this performance is to be manifested. (2) the learner's 
initial capabiH.ics relevant to for coming instruction are assessed, n^ 
educational alternatives suited to the student's initial capabilities are 
presented to him; tl.e student selects or is assigned one of these alter- 
natives. (4, the student's performance is monitored and continuously 
assessed as he learns. (6) instruction proceeds as a function of the 
relationship between measures of student performance, available ins.ruc- 
t.ona, alternatives, and criteria of competence, and (M as instruction 
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proceeds. x4ata'ar'c ^encfalcd for monitoring and in^proving the instruc- 
tional sy.sti'in. 

Ilium t oiTipononl^ of tho IPI program include the Individualized 
MalhcMiuUus Curriculum (I.incK'all ^ Bolvin, 1967), the Individualized 
bticnie Curruulum (Klopfur. i97Z). and the New Primary Grade Reading 
Sysicni IBc'ck, l^^TZ). Curriiuluni coinponenlrf for PEP include beginning 
math, i lassification and i omnuinicalion skiIIs, perceptual skills and the. 
exploratory li^arning skills (Ursnick. Wang. Rosner, 1973). However, 
not ail of the loniponents listed above are used in tho LRDC Follow 
Throtiiih schools,. 

Aspects of curriculum developed for each of the cucricular com- 
poncMits mi hide: tlic ifualion of cu rriculum objectives, the sequencing 

ol till* obnM tivc'>, {Yif design of instructional and learning activities and 
materials, the specification of teacher and student be lia viol's, and the 
spec ilitaiKiii of procedures for diagnosing and monitoring student learning 
prouress. [Provision for the diagnosi^ and monitoring of individual stu- 
dent .s iearmng progress is at the core of the individualized instructional 
prt»uranis. Prc»c edu; s and instruments (e.g.. Cox, I96b; Wang. 1968) 
for (hagiKjsing and monitoring student learning has been designed to pro- 
vide le.ic lie rs witli the informatic»n necessary for adapting the use of the 
prcigram loinponents in the individual Students, and to communicate, on 
a substantive basis, vvith parents and others concerned with the learning 
progress and Ihe fle\elopment of the sturlent. 

The iniplemeiilation ot the 1JU)C Individualized Instructional Pro- 
grams in classroom settums, ideally requires two adults in each class, 
a teacher and an aide. The adults during the instructional period, 
generally perfc»rm two basic regies, Ihe "traveling" role anrl the "testing 
and tutoring ' role. The traveling role requires the teacher to circulate 



13M 



\ 



BEST COPrN|iVAILABLE 



anumu the stu.l.MUs. hHp.nn w.ih their luarn.nn i^^.^s^^d checking Ihcm 
of, an llu-y arc ...mpU-t.nK and uueracting in various w\ys, generally quite 
bnc. n,.- u.si.nK and storing role requires the ,oache\to worl< inlen- 
s.vHy will. .nd.vHluals or s.nall groups of students for suc\purpose» as 
adn>in.HU.r.n« d.a«n.,s.u tests, tutoring individual students.Vving group 
U ss.ms or working svith a group of students on a speciul lea rnW project. 
Tl... rol.-s d. s. ribed above are -ideal. zed" descriptions; in pra^i^cc. how- 
..ver. .he two adults flu. tuale from one role to the other as need avnses. 

llu-re are iw.. b.isi. sets of tcacl.. r functions, both nocessar\£or 
SM.uotI, and effe. t.ve i.uplementat.on of the LRDC program in classroi^^ 
sellings, the .uananemcnt functions and the instructional functions. Th" 
.uananement fun. t.ons are concerned with the establ.shment of an effdctiv 
system for c lassroom nmnagemenl. They include such functions as: the 
prov.H.on of materials and equipment for the various components of the 
progran,. their physical arrangement, display and sto rage , maintenance, 
as well as demonstrating and explaining rules and the use of materials, 
and praismn or otherwise reinforcing students for appropriate self- 
hi.inagei}>ent a. it ivilies. 

Two sets of teacher instructional functions have heen identified: 
,1,.. ■■di-da.-li. •• and the ..nsultanf functions. The didactic instructional 
„.„. ti.ms are related lo the administering of tests associated with the 
...i-mal .urri.ula. prescribing learning tasks on a daily basis, checking 
p,.es. np.ive activities and giving help on them as required. The teacher 
,,,,/„, ,,u. aide, also assume, under the didactic instructional functions, 
„„. i...s,,nns,b,li.y ... .ondiic. special tutoring sessions on certain specified 
. ,ives. as well as larue or small group lessons as dictated 

In the v..noMS .urricula and by the needs of the students. 
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The consultant tundions are less highly stnu tiircd, but like the 
didactic functions, ihey are carried out in the i ourse of both travehiij* 
and testinu and tutoring. These 'func tions require the teat hcM' and/or the 
aide to focus on observation of students' learning processes beyond what 
is provided in the formal tests; use questioning and probing techniques 
to stimulate development of sel f- ref lecti ve in problem-solvmg activities 
on the part of the students: er^age in planning with studc>nts. helping 
them decide what to do and how to do it; pose problems for students to 
work on and help them in planninu and carrying out solutinns; and engage 
in games and other forms of play with the students. 

No clear distinction can be made in pr ictue between management 
and instructional functmns- -e ve ry act contributes to both. Similarly, 
the teacher should fulfill both didadic and consultant instructional roles. 
Nevertheless, the distinctions are useful as a mcMns of describing the 
range of functions that teachers must meet when implement ing/he IMDC 
individualized Instructional Programs, The distinctions between tho two 
functions also serve to cha racter ix^c, in a general way, the teacher behav- 
iors to b^_' observed in an LRDC classroom. 

The role of the student under the LRDC Indivuhial i /ed Insi rut tinnal 
Program centers around the management of one's own activities in the 
learning situations. In general, the student is expected to: 

1. Work and complete certain tasks prescribed by the teacher Uhe nature 
and the amount varies from student to student, and depends on thr 
learning needs and the individual student t ha racte nslu s ), 

2. Work and complete certain tasks ui the stucU-nt's own dunce. 

3. Make decisions about when to do what work (the range of the options 
and the degree of control varies from age to age and c lass to c lass). 

4. Take diagnostic tests when asked by the teacher. 

5. Participate m tutoring sessions when asked by the tearher. 
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Part.c .paic .n group activilios wlu-n roq»u-< ci. 
7/ AHk Iho tcachor to Cck the work one .o.upl.M.-s Uu- ass.gnments. 
H Ask for help (from the teacher and/or p.-cr., uh.-n noo.led. 
... Ass.Ht others (m.l.at.^ and/or when recucstcd, for n>ana,,cn.en. as 

well as for learning purposes. 
10. Follow classroom management rules. 

, I. Locate learning materials and equipment independently. 
U. Carry out material management responsibilities (e.g. • ^lean up. 

relurn ct^upment, etc. ). 
, , Take turn« and share activities and materials with others. 
,4 interact w.th peers for personal as well as school related activUlcs. 
Tolerate disruption to the activities at hand for attending certain 
group activUies and/or certain testing or tutoring sessions. 

16. Attend to the task at hand and ignore distraction from the different 
activities being carried out by others at the same time. 

17. Budget one's ow-n work time to meet the time constraints established 
for icTtain tasks. 

The student roles Hated above are behaviors required to function 
e(fec tively under the LRDC Individualized Instructional Progran.s. How 
ever the ability to carry out the roles are not assumed as part of the 
entering behaviors of all students. Students are taught to acquire the 
„..nimum level of competencies required to assume these self -managen. 
an'l independent learning roles. 
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Obser.vational Instruments and Data Recording Forms 

Used in 

The Differentiated Child liehavior Observational System (DC13) 

by 

Sylvia Ross and Herbert Zimiles 
Rank Street College of Education 
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APPKNDIX B 
nCB Form 



Bank Street Follow Through 



Date . 



Time from. 



Community 
to 



OCB (Group) 



(5/1/71) 



School 



No. of Children 
No. of AdulU 



leacher 



, Activity 



No. of Children , 



Adult Role 



GI VES INFORMATION RE 
Identity, Situation 
Prediction, Plan 



Fu nction. Process, Directions 
DifferentiatinQ Properties 

Relationships 

Category, Class 



Casual Reasoning. Problem Solving 
EXPRESSES 



Routir>e Need 

Preference. Desire 

Feelings. Attitudes 

Nee< js , 'Task 

Needs • Social/Physicai 



Affection, Warmth, Humor 



Concern for Others, etc^ 



Lack of Concern for Others, etc. 



^HQWS AUTONOMY' 



Records Chotte of Task 
initiates Ya»^ 



Initiate^Attentional Focus 



Seeks Answers (in Book, etc.) 



\^ Selects Materials, etc. 
\ Replaces Materials, etc. 



Straightens Up Work Area 



Resolves Conflict 



Enacts Transitional Move 



• situation appropriate, no adult intervention 



ASKS QUESTIONS RE 
Identity, Situation 



Prediction, Plan 



Function. Pfo&m, DIfectiofis 



Differentiating Properties 



^slstionships 



Category, Class 



Casual Reasoning. Problem Solving ' 



BE HAVES AGGRESSIVELY 
Physical - Initiates 



Ph wsical ' Defense 



Verbal Inititates 



Verbal - Defense 



Takes Others Belongings 



Abuses Material - Equipment 



D^roys Another's Work 



Ci allenqes Established Classcoom Limits 



COMMUNICATES VIA SY|<60UC 
PLAY AND REPJ^ESENTATION _ 



Directs Group (Dramatic Play, etc.) 



Ptojectt Fantasy 



Mak es Descriptive Comparison 
N arrates^^torv. Sings Song, etc. 



Tells Original Story. Plays with Words. 
Sounds ; ' « 



Shared Reading Experiente (Reads 
Aloud or Shares Rtctures) 
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Appendix B (Cont^d) 
Bank Street Follow Through 

CHILD BEHAVXQR SCANNING FORM' 



1/18/71 



Community. 

Grade: 

Teacher: 



Obc«rver , 

Date: 

Time: 

— Context:. 



IT 



.To:. 



Children 


1 


2 


3 


4 


S 


6 


7 


8 


9 I 10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


23 


23 


24 


25 


26 


27 


28 


29 


30 


1. h/on Verbal B 












































11 Inv 






1 






1 


1 

i 




























! ! 














1 2 Sel 






— i — , 


































1 
















1.3 Org 












r 




































1.4 Ret 






































"1 - 


















15 NOF 




' — 
























































16 Soc 






























































1.7 Ab 




























































2 Verbal B 




























2 1 Cx 


































1 


























22 Q 




i 

' — ? 






















































23 RQ 






























































24 Read 




















































•2 5 VA 































































List all Activities: 



*To be used with DCB form. 
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Appt*ndix H (Cnul'd) 



C HILD BEHAVIOR S CANNING FORM (to be used with DCB) 
: Directions and Categories 

Directions. A separate entry is to be made for each child in the ciassroom with the 
~* entire group represented on edch scanning form. 

Start with the child nearest the door then continue in a counter-clockwise 
direction until the entire group has been scored. 

Look at each phild long enough to categorize his behavior. It should take 
from two to five minutes to score the entire group. 



} Non-Verbal Behavior 

1.1 INV . Involved: tesk oriented 

1.2 SEL Selects materials, equipment, etc' . . 

.13 ORG Organizes, i.e., arranging materials, furniture, etc; directing others 
' in relation to position in physical setting. 

1.4 RET Returns or replaces materials, etc. 

1.5 NOF No observable focub. e.g., daydreaming, withdrawn. 

1.6 SOC Social interaction (non task-oriented). 

1 7 AB Abuses materials, equipment, etc; destroys another's work; physically 

abuses another. 

2. Verbal Behavio r 

2 1 C X Comments, explains 
2 2 0 Questions 

2 3 RQ Responds to questions 
2 4 READS Reads aloud 

2.5 VA Verbally aggressive, i.e. treatens another 
r I. BOY B (alone). 0 (with or to adult); A l^'^h or to another child) 

— GIRL ■ G (alone). I (with or to rdultl; /h (with or to another child) 

Context Describe period m general, o.(j Discussion, work period . 
SuUHHt Area. After completing observation list each activity noted. 
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^^v^Apprnclix B (CoiU'fl) 

^"^"^"'^'^V . : ^ Teacher 



Observer 



Date 



SUM MAR YSCALj: CHILD BEH AVIOR (to be used with DCBGroup) 
Items to be rated on scale of 1 jicw^st rating) to 4 (highest); Circle Number 

1 txtent to which children participate with interest and enthusiasm. 
12 3 4 



2. Extent to which children show ability to sustain attentional 
focus and remain involved in activities. 

12 3 4 



3 Extent to which children remain at tasks until completed. 
12 3 4 



4 Extent to w)nch actingout dtsruptive behavipr is evidenced, 

12 3 4 

5 fcxtnnt to which withdrawn, unrelated behavior is evidenced. 

12 3 4 
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Appendix B (Coated) 



Bank Street College of Education 

Slowest 112th Street 

New York, Neyy York 10025 



DCB OBSERVERS 

Interview with Claaroom Team 
Ob«rver- I've enioy«l visiting .his Cas^oom very much and would liKe to asK you a .ev. .ues.ions: 
1. Did you do anything special today? — 



2. In general, was this a typical da^? 
If not how was it different? 



3. Did the children behave pretty much as they usually do? 
If not, specify: - — 



4. Did each of *e cNldren who were observed individually behave as he/she usually does7 



If not. specify. 

i *k;m. Comments: 
Name of child: 



4 ^liA' Comments: 
Name of child. 



, ' School 
Community — 



Date 

Observer ^ 
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% 

lusM-Mi M>ns for (JbsiTvalional Techniques 
used in ihf* sttidy? 

Obsrrvaiion as a Tool for 
Evahiation .of Implementation 

by 

Caea l.einliardt 4k 
J.t^arning Kosoai ch and fVvelopmenL Center 
University of Pittsburgh 
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APPENDIX C 



Instruc 



tLons for Que8aons_6^nd_7 



f.vo enrolled children should be present in the room, 

l^rxhT:; o^C sLou^ started Lf three o. more children have 

eadv St er^orkLng Ln the exploratory area (u.less exploratory .s 
always conducted simultaneously with prescription work). 

( nraw a auick map of the classroom. Divide the exploratory 

from the prescription and «ive a general idea in the prescrip, 

on area of desks or tables, library, shelves, easels. Jin- 
dows btackboards. audio equipment. TV. and where children 
arl sitting. Also show in a second color the way you dmded 
the room' Then for three minutes, trace the walking path o£ 
the teacher with a solid line. 

r 

Indicate the verbal contacts (when he/she is not physically 
pres" ) with a dotted line. If the teacher has traveled -P.dl 
around the room so that your diagram starts to get messy 
aher two minutes, sLop.and note on the side tha) you only 
recorded for two minutes. See example below. 



r: "i,",." ;r;:.::. ~™ « » — 

the iPjCM observed in 6b 



__- ■ P,e«npt.on Area De5k» of Tdbl«. Sheives. EaMlv Wi. dov« 

,„c,.d, I^P'--;'^-,,^ TV. App,o«.ma.. .oc...on of ch.ldren. 
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* 



^»b. Using the form provided, mentally divide the classroom into 
six areas using the most natural division possible. That is, 
divide the room by the way the children and desks arc located 
rather than by equal spaces. Record in each cell whether that 
cell if? a prescription area (P) or exploratory area (E) or both 
(PE). At hp be^inninj^ of a ten-minute observation, count how 
many children are in each cell and record that number in the 
second upper left hand cell of each of six blocks. Count again 
at ihe end of the observation time and record the number be- - 
side the first. Children will move around during the 10 min- 
utes of recordinj4: however, the average number per cell 
should remain relatively consistent. A contact is considered 
any verbal or physical statement or touch made by the teacher 
to oner or more specific students: it includes general class 
statements such as; "We are all working well today, " or "it's 
«e!tin« a litth^ noisy," etc. Everytime a teacher makes a con- 
fact, code it in the block to which the contact was directed, 
K e. . if the teacher iy standing in the upper right-hand corner 
of the room and speaks to a child in the middle left block of 
^ the room, record the contacts in the latter block, not the 
former. The exception is general contact^; they are recorded 
in tht' cell where the teacher stands. The verbal behavior of 
the teacher should be recorded for IS consecutive minutes-- 
startlng at least five minutes after the children have started 
math work, 

^»r. If the teacher is traveling with her aide, use the same way of 
dividing the room and record the aide's travel for the next 
10 minutes in the same way you did for the teacher in (>b; if 
n«)t. continue with th(» teacher, (See example below for 
questions (>h and 6c. ) 
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6b & Type* oi leac^er cooucn toi IQm nutf^^ MCX 

m m 



It teacher iraveiinq alone 
Number of aUultv trjv«lirii|. 
»t*fted 



Time obvervJtion 



The Codes are: 

M Management - - A management comment is differentiated from a cog 
nmW^eln that it is not subject matter oriented and deals with th 
overall behavior of the child in the classroom, such as raising or 
folding hands, sitting at desk, getting box or ticket, etc. In addi- 
tion. there are management behaviors which relate to the perform 
ance of a cognitive task such as getting the materials out, turning 
to page X. writing the answer on the dotted line. etc. Comments 
wh.ch deal with these types of behaviors are often accompanied by 
additional or further comments which are cognitive anS are coded 
as such (see example below). 

C Cognitive --This is a contact which deals directly with the subject^ 
matTirc'ontent. or the acquisition of subject iViatte/ such as sound 
ing out words or counting out chips. 

, Positive ve rbal praise -Usually this includes an ad^jective such as 
g^w^erful. etc. . but not necessarily, but obvious tone of V( 
changes which are positive can be coded as such. P°";r'"„^*"J 
used in combination with management, cognitive, or both togethe 
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r o sfon P"""''^'" Ji-££Sl']?eni- -Usually this includes a command 

o s op someth.n, or shoutinM or speaking with raised voice-Tan be 
used a one when , he behavior is no. identified in the statement (see 
oxan.pIe, or w.th management, cognitive, or combination st'ement 

^ J.udon?.'' Vr'^" '"'''^'"^ ''^y ^ ••^*-*'« hand or formal 

studcm-,..acher s.^nal). This does not include eye contact or be^av 
.oral ,„.ua,.on sueh as misbehavior, bu, it does include arm tapp'nl 
Ct::,' " ' " • ^"^^ ^-"^^^^^ physical signal ' 

X For uncndeubio or unheard statements, try to use only rarely. 
Teacher says nothing - -and checks off work. 

ove_rlo.ie_s. such as ' Cireat. you road aTritrpaTe—aTslfn^f 

^;;-rir'-----J^='r^'^^^^ and co.ni- 

readin«:1^xtTwo-rir^-the 

-MC AiLii^ralJ_n.:^.n .v.._unj..^^^^ ,he management or the 

such as -M .c or -M. .s possible, but I have never observed 

1" ^"ntact is ne>,atn'e when .he teacher tells a student to 

s.op behav.n. in on. way or ano.her. neutral If behavior is requested 
and pos.,.v. .f U.ha v.or .already done is praised. The decis^onas 

•so. whether .1 refers to cognitive or management behaviors. 

7. On your observation sheo. ob. you will see four X's. The 

l.u a.ion of .he X's is .he location of the child you will inter- 
v.ew. Iha. is if an X appears in the upper right hand corner 
of ,l,e ,h,rd cell, f.nd .he child who is in the upper right hand 
corner of .ha, cell in .he class. Circle the sex of the child 
on .he form and and request to speak to him or her; if they 
refuse, record .( as such and ^o .o the next X. ' 
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^ ^ «;ria and two boys. I'hat means, 
YOU should interview two g.rls and two b Y ^^^^^^ 

U your first two chUdren a 6 gxrls then^ ^^^^^^ 

be boys closest to '^e X. etc^ comple^ sentences-, and 

dren's responses in list form-not """PJ 

onlv a«k the additional questions if the chile oes n 

"ose areas by him/herself. (See below for example of 

child interview questions.) 



rh.id interview (2 boys and 2 girls 
rantlomlY selected) 

7 What do YOU do during PEP or IPI malh ? 
How do YOU qei YOur assignment ? 
How do YOU know what lo do ? 
Do YOU ever write YOur own prescription ^ 

What hapiiens when you finish all of your 
work on your prescription sheet ? 
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SMOS Form and Definitions 
used in the study, 

The Use of Observational Data for Formative 
Kvaluation of an In?>triicliona! Modi^l 

Margaret C. Wan^ 

Lt•arnln^ Kosearch and Dcvrlopnunt Ct-nfer 
University of PiUHburuh 
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Siudent Behavior Observa^ion^c^oduM^^ 

Time Iniorvals 



Vur tables 



1 AcLivili.iyfie5 

b. ^M'oraiof^ ^ - ' 

c No acuvjiy - 

2 Scttifig 

a. pTPUfi intcriictivi! — — 

b GiQup parallel - - 

c. Individual . ^ . . - - 
d No activity _ - 

3 Activity outcome 

a r.ompjetu activity 

b Loavo5 task iincompletc] _ _ . 

4 Manner in which activities arejnUiated: 
a Assiigiied 

b Inyitatiun by another student ^ . . 

c, Sell initiated 

d. Not sure 

5 Manner m which activities arc carried out 
a Aimless 

b Inattentive 
c. Purposeful 
d Waitimj 

6 Inter aclioi»s with ti'acher 
Tyj>c 

a Student iniTtated 
b Teacher initiated 
c Verbal 

d Nonverlwl . 

Purpose 

a. Mand(jeinpni 

b Inslruchonal 

c. Personal 

d Cannot determine 

; Interaction with students 
Tyix- 

d Student initiated 

h Initiated by anothnr student 

(1 VHrt)ai 

d Nr)n vMhjl 

Puri«>v' 

d Sharing ulfMs^ injfr.ah and activities 

b DisaijrfM'nefJt 

<; Corwi'lstition 

(t S»'ek »n1(>in^atMH> or he'.) 

e Asstvl oth«*rs 

I Cannot d«'ti»rrn»ni» 
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D efinition of Variables Included in the SBOS 
l« Activity types 

^* Prescriptive- -activities assigned by the teacher. These include 
tasks listed in students' prescription tickets, as well as on-the- 
spot assignments made by the teacher* Prescriptive activities 
are generally prescribed to teach specific skills, based on diag- 
nostic tests of the LRDC individualized curricula, 

^' ^^xploratorjr- -activities included in the exploratory component of 
the program. They generally include activities in creative arts, 
block construction activities, and extended activities that are 
related to such subject matters as science, math, reading, and 
prereading, 

2» Setting 

^^j*Q^P-l^^^r ^ct ?ye- -eniia^inu in an activity, among two or more 
students, sharing a common goal(s) and/or idea(s), 

^* Group paran >?l~»>enaauinH in an activity among two or more stu- 
dents, without sharing a common goal(s) and/or idea(s). A paral- 
lel group refers to location. For example, in block construction 
two students may be sharing blocks and sitting next to each other, 
I'Ut they are building different structures, 

c. Individual- - student WQrks alone. 



^' No acti vity- -student is neither working on individual prescrip- 
tive^ exploratory, nor group activity. Behavior such as clean 
• up, waiting to be dismissed, going to lavatory, are considered 
under the "no activity" category, 

3, A ctivity outcome 

a. C ompleted ac tivity- -(1 ) prescribed activity is recorded as com- 
pleted when student actually puts the task he finished back on the 
shelf, and checked off by the teacher; (2) an exploratory activity 
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is recorded as completi J when the student has completed the 
activity and when the teacher has checked off the activity on the 



exploratory ticket. 

b Le aves task (incomplete)-a student is recorded as having left • 
rhe task incomplete when he leaves his prescriptive or explora- 
tory, or group activity for more than an observation interval, 
or engages in a different activity before completing the task at 
hand* 

4. M anner in which activities are initiated 

a. As8igned--an activity is recorded as assigned to the student if, 
the teacher directed him to do it. 

b. Invitat ion by others - -an activity is recorded as an invitation by 
otl^lf the activity was suggested by another student or t^.che 

c. Se lf -initiated- -an activity is recorded as self-initiated if Uie 
student chose to do it on his own. 

d. Not sure- -an activity is recorded as "not sure" when the observ 
dlT^^atch the manner in which the activity was initiated. 

5 Manner in which activit ies are carried out 
• ■ " p 

a. ■ Aimless (wandering around without any activity)--a student's . 

b^^w7r is recorded as aimless if the student wanders around 
in the classroom not sure of what he wants to do or what he 
should be doing. 

f 

b. Unattentive (wonder, watching, wljile engaged in activity) -a 
student's behavior is recorded as unattentive if he stares into 
space, interacts with other students (verbal or nonverbal) on 
matters not related to the tarek at hand. 

Purposeful- -a student's behavior is recorded as purposeful if 
hi is pacing attention to what he should be doing. 
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B£sr mmmu 



^« Waiting- -a student's behavior is recorded as waiting if he is 
' sitting or standing with his hand raised for the purpose of getting 

the'teac^er's attention to check off his assignment, or ask ques- 
tions! oeek help, etc. 

^* I nteractio ^ with teache r ^ 

a. Type ; 

^^Mden t initiated - -an interaction is recorded as student 
Initiated if the student began the interaction either by 
speaking first or raising his band. 

Teach er init i ated --an interaction is recorded as teacher 
initiated if the teacher began the interaction* < 

(3) Verbal- -an interaction is recorded as verbal if the student 

' " "" ' ' • ♦ 

initiated or responded by speaking. 

Nonverbal - -an interactic;-* is recorded as nonverbal if the 
student responded or initiated physically such as nodding 
his head, laughing, or fighting without verbalizing. 

b. ^ Purpose; ^ 

Manageme nt - - an interaction is recorded as management if 
the purpose of the interaction dealt with noninstructional 
aspects, interactions not related to "academic" learning, 
such as asking and answering routine clas^oom manage- 
ment questions are recorded under this category* 

In strnctional - -an interaction is recorded as instructional if 
the purpose of the interaction is related to either prescrip- 
tive or exploratory activities. Interactions such as checking 
work, asking questions about student's learning task, tutor- 
ing, demonstrating, testing, and explaining are recorded 
under this category. 

l^t^rsonal - -an Interaction is recorded as personal if it is 

related to personal matters (e. g, , How do you feel today? 

Is your brother still in the liospital What a pretty dress!)* ^ 
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7. Interactions y/ith peers 



, ' t a. Til>e 

i (1) 



Student initiated - -an interaction Is recorded as stUdent ini- 
tiated if the student up !er observation began the interaction, 



(2) Initiated by another student - -an interaction is recorded as 
' response to other student if the student under observation 
enter's in^o an interaction which another student initiated. 



(3) 



Verbal- -an interaction is recorded as verbal if the student 
under observation responded or initiated by speaking. 



(4) Nonverbal - -an interaction is recorded as nonverbal if the 
student under observation responded or initiated physically 
such as by pushing other students, fighting, smiling. laughJ 
ing. etc. 



b. Purpose : \^ 



(1) S haring - -an interaction is recorded under sharing when the 
interaction involves sharing ideas, materials, activities, 



etc. 



. (I) Disag^ree8--an interaction is recorded as disagrees when 
the student under observation is either arguing or fighting 
with another student. 



(3) 



(4) 



Co nversation - -an interaction is recorded as conversation/ 
when the student under observation is Ulking to another / 
student about personal matters not related to school wor^. 

Seek information - -an interaction is recorded under seek 
information when the stude-t asks other students to prbvide 
information on task related or nontask related matters. 



(5) Assist, others --an interaction is recorded as assisting 
others "vhen the student under observation offers ahother 
student help or advice on manageme nt or instructional- 
learning matters. / 
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Criteria for Observations Form 
used in 

A Developmental Study of Group 
Formation in Children 

/ 

Donald M. Omark and Murray S, Edelman 
University of Chicago 
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Criteria for Observations 



N1 



N2 



Distar^cf N3 , 

Cover u'* 

Other 



Verb. 



Agg. 



Imit, 



Int. 



Comments 



txplanation 
X 

Distance Covered 
Nl. N2. N3 
Distdocf* 



Vi?rb 



Irnit 



Int 



The child bemy observed 

child during Ihe 30 seconds he is observed. 

These are the three nearest neighbors, spatially/to X. They may or may not 
be interacting. 

This ,s the distance «hich each is from X This «i|. be recorded as: 
I C actual contact wlh X 

; T . Within touching distance if both N. and X raised the.r arms, 

but not touching 
( 1 -^m I S normal speaking distance. 
( 2m.) Y yelling distance. 

, .» X nr N IS talkmg dDring the observation period. M 
^Jc S rc rniltL^'crn'L'detJmined, an arro. -s inserted in the bO. 
Z '"^ans X talks to N = means both are talking to each other. 

A check here means that an obvious physical aggressive 1"-°;';'; '^^^^^ 
eg , hMting, punching, pulling do^vn If the fighting is one sided, an arrow 
indicates the attacker 

A check he-e indicates imitative behavior, e.g.. two or more engaged in the 
sjfpe kind of action .it the same time 

perniit later hypotheses generatKjn. 
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